Skip to content

“Production answers don’t match what we tested.”

RAG answers that change between testing and production

When retrieval settings live in five services and answers carry no source, nobody can tell why an answer changed. DocSlurp keeps one pipeline and makes every answer checkable.

Why it happens

  • Each client sets its own retrieval options, so the app, the bot and the agent each search differently.
  • Scores that only rank hits within one query make thresholds tuned in testing drift in production.
  • Answers without sources cannot be checked, so regressions surface as complaints.

What changes with DocSlurp

  • One pipeline behind every API

    REST, the SDK, the CLI, MCP and both chat endpoints use the project’s retrieval defaults. A request changes only what it passes.

    Details
  • Relevance you can compare

    Each hit’s relevance measures the match itself, not its rank among the other hits, so one threshold works across queries.

    Details
  • Every answer cites its page

    Search hits and chat answers name the file and page they came from, so people and agents can open the source.

  • Usage on every response

    Search and chat report their tokens and cost, so a change in behavior shows up in the numbers.

chat.ts

const { answer, citations } = await client.chat({
  q: 'Which supplier contracts auto-renew this quarter?',
});

Sound like your setup?

Tell us what you run today and what it costs. We’ll say plainly whether DocSlurp fits.

Jump to