Skip to content

LearnGenerate & answer

Agentic RAG

Iterative retrieval · Search agent

RAG where the model decides what to search for, reads the results and searches again until it has the evidence it needs, instead of retrieving once up front.

In DocSlurpGlossary entry

On this page

Agentic RAG lets the model run the search instead of receiving a single retrieval. It gets a search tool, reads what comes back, decides what is still missing, and searches again, until it has enough evidence or runs out of budget. Then it writes the cited answer.

Plain RAG retrieves once, using the user’s words. That fails on questions whose second half depends on the first. “Who approves the jet for the exec leading the London launch?” needs two lookups: launch-review.pdf names the executive, then travel-desk-guide.pdf names who approves that executive’s trips. The first search can’t contain the executive’s role because nobody knows it yet. An agent reads the launch review, learns “the COO leads the London launch”, and writes the second query itself.

The loop is tool calling in a cycle:

  1. The model sees the question and a search tool.
  2. It calls search with a query it chose.
  3. The results come back as tool output; the model reads them.
  4. It calls again with a sharper query, or answers.

The gotcha

Every step resends the whole conversation, including all earlier results, so cost grows faster than the step count. Bound the loop: a step budget, a cap on tool calls, and a rule to stop when a step finds nothing new. Without a stopping rule, agents keep searching for evidence that isn’t there instead of abstaining. Prompt caching softens the repeated input cost but doesn’t remove it.

In DocSlurp

In DocSlurp

Chat runs a bounded search agent by default (DOCSLURP_CHAT_MCP_ENABLED): 4 model steps, extended to at most 8 only while each tool step finds new evidence and input stays under 150,000 tokens (core/packages/shared/src/project.ts, core/server/src/services/chat-mcp-agent.ts). Filtered questions and tool_choice: "none" use one direct retrieval instead.

Sources

Jump to