Agentic RAG
Iterative retrieval · Search agent
RAG where the model decides what to search for, reads the results and searches again until it has the evidence it needs, instead of retrieving once up front.
In DocSlurpGlossary entry
On this page
Agentic RAG lets the model run the search instead of receiving a single retrieval. It gets a search tool, reads what comes back, decides what is still missing, and searches again, until it has enough evidence or runs out of budget. Then it writes the cited answer.
Plain RAG retrieves once, using the user’s words. That fails on questions whose second half depends on the first. “Who approves the jet for the exec leading the London launch?” needs two lookups: launch-review.pdf names the executive, then travel-desk-guide.pdf names who approves that executive’s trips. The first search can’t contain the executive’s role because nobody knows it yet. An agent reads the launch review, learns “the COO leads the London launch”, and writes the second query itself.
The loop is tool calling in a cycle:
- The model sees the question and a
searchtool. - It calls
searchwith a query it chose. - The results come back as tool output; the model reads them.
- It calls again with a sharper query, or answers.
The gotcha
Every step resends the whole conversation, including all earlier results, so cost grows faster than the step count. Bound the loop: a step budget, a cap on tool calls, and a rule to stop when a step finds nothing new. Without a stopping rule, agents keep searching for evidence that isn’t there instead of abstaining. Prompt caching softens the repeated input cost but doesn’t remove it.
In DocSlurp
In DocSlurp
Chat runs a bounded search agent by default (DOCSLURP_CHAT_MCP_ENABLED): 4 model steps, extended to at most 8 only while each tool step finds new evidence and input stays under 150,000 tokens (core/packages/shared/src/project.ts, core/server/src/services/chat-mcp-agent.ts). Filtered questions and tool_choice: "none" use one direct retrieval instead.