Backpressure
A busy system pushing back on its callers, by making them wait in a bounded queue or refusing work with a retry hint, instead of accepting more than it can finish.
In DocSlurpGlossary entry
On this page
Backpressure is how an overloaded component tells its callers to slow down. Without it, work piles up in an unbounded queue. Each new request waits behind all the others, latency climbs for everyone, memory grows, and eventually something times out or crashes. The queue made the overload later and worse, not smaller.
With backpressure, the component keeps a bounded queue. When it is full, or a request has waited too long, it refuses new work immediately, ideally with a hint for when to try again:
HTTP/1.1 503 Service Unavailable
Retry-After: 5
The requests it did accept stay fast, and callers can retry, slow down or go elsewhere.
In practice
DocSlurp’s hybrid search shows the pattern. One search makes many database round trips, and a burst of chat agents can fan out dozens of searches at once. Instead of letting them all wait in the database connection pool, where they would block logins and uploads too, searches wait for a slot in a small per-process limiter. Its queue is bounded and time-limited; beyond either limit a search fails with 503 and Retry-After. Waiting searches are served round-robin per tenant, so one organization’s burst waits behind itself rather than in front of everyone else.
The gotcha
Backpressure only works if callers respect it. Clients that retry instantly in a tight loop turn a short overload into a sustained one. Retry with exponential backoff and jitter, and honor Retry-After. Rate limiting is the related, per-caller control: it caps one identity’s request rate even when the system has room.
In DocSlurp
In DocSlurp
Hybrid search runs behind a per-process limiter: excess searches wait in a bounded queue, served round-robin per tenant, and fail with 503 plus Retry-After when the queue is full or the wait times out (core/server/src/search/search-limiter.ts). Chat turns have an in-flight cap per key (core/server/src/middleware/chat-concurrency.ts).