Skip to content

LearnRetrieve

Conversational retrieval

Conversational search · Follow-up question rewriting

Searching on behalf of a chat turn whose meaning depends on earlier turns, usually by rewriting the follow-up into a standalone query first.

In DocSlurpGlossary entry

On this page

Conversational retrieval handles search inside a chat, where a message often only makes sense with the turns before it.

User: When does the London office open? Assistant: In Q2 2027, with first-year bookings forecast at $6.5M. [1] User: What about Tokyo?

“What about Tokyo?” is clear to a person and useless to a retriever: embedded alone, it finds passages about Tokyo in general. The search that the user means is “When does the Tokyo office open?”

There are three common approaches:

  • Rewrite the follow-up. A model reads the recent history and writes a standalone query, sometimes called question condensation. Research datasets such as CANARD were built to train exactly this rewrite.
  • Search with the history. Concatenate the last few turns into the query. Cheap, but old topics leak into new searches.
  • Let the model call search. In agentic RAG, the model sees the conversation and writes its own search queries through tool calling, so the rewrite happens implicitly.

The gotcha: the answer prompt has history, the search doesn’t

Many systems pass chat history to the answering model but search with the raw latest message. The model then sounds context-aware while answering from the wrong passages. Check what string actually reached the retriever, not just what the model saw.

In DocSlurp, the default chat agent receives the earlier turns and writes its own queries. A direct answer (agent: {tools: false}) searches with the latest message as typed and gives the history only to the answering model, so a follow-up there should be phrased as a full question.

In DocSlurp

In DocSlurp

In the default search-agent chat, the model sees the earlier turns and writes its own search queries (core/server/src/services/chat-mcp-agent.ts). Direct answers (agent: {tools: false}) search with the latest message alone; history reaches only the answer prompt (core/server/src/services/project-chat.ts).

Sources

Jump to