What is Agentic RAG?
Agentic RAG is a form of retrieval augmented generation in which an autonomous AI agent decides when to retrieve information, what to retrieve, which source to pull it from, and how many times to repeat the process before answering. It builds on traditional RAG, introduced by Patrick Lewis and co-authors in their 2020 paper on retrieval augmented generation for knowledge-intensive tasks, which converts a query into an embedding, retrieves the most relevant document chunks in a single step, and conditions the language model on those passages to produce a response.
The key difference is control. In traditional RAG, retrieval is static and single-shot, so the system passes the raw query to one vector index, takes whatever comes back, and generates an answer with no way to check quality or recover from errors. Agentic RAG places a layer of agent reasoning on top of that retrieval mechanism and treats retrieval as a tool the agent chooses to use based on the complexity of the prompt. Instead of retrieving once and generating once, it retrieves, grades the results, retrieves again if needed, and only then generates.
In practice, this means an agentic system can break a complex question into smaller sub-queries through query decomposition, run those searches in sequence or in parallel, and use a semantic router to send each sub-query to the most appropriate destination, whether that is a vector database, a relational SQL database, or an external web search. It evaluates the retrieved chunks through a dedicated grading step that judges whether the information is relevant, accurate, and free of contradictions before anything reaches the generation phase. If the primary source returns weak results, the agent can fall back to alternative tools rather than producing a confident hallucination.
This self-correcting behavior is what most clearly separates agentic RAG from its predecessor. Frameworks such as Self-RAG train the model to decide for itself when external knowledge is needed and to critique how well retrieved passages support its answer, while Corrective RAG adds an evaluator that classifies retrieved documents as correct, incorrect, or ambiguous and triggers a fallback search when the evidence falls short. The result is a dynamic retrieval loop rather than a fixed pipeline, which makes agentic RAG well suited to complex enterprise workflows, multi-document synthesis, and cross-referencing data from sources that change quickly, though it comes at the cost of higher latency and more variable token usage from the extra model calls.
