Rate limiting
Capping how many requests an identity, such as an IP address, user or API key, may make in a time window, and answering 429 when it goes over.
In DocSlurpGlossary entry
On this page
Rate limiting caps how many requests one identity may make in a time window, for example 60 searches per minute per API key. Over the limit, the server answers 429 Too Many Requests and says when to come back.
The point is fairness and safety more than raw capacity. Without limits, one buggy script or one leaked key can consume a shared system’s database, model budget and workers, and every other customer pays in latency.
How the counting works
A fixed window counts requests per calendar minute. It is cheap but lets a caller send double the limit across a boundary: 60 requests at 12:00:59 and 60 more at 12:01:00. A sliding log stores every timestamp and is exact but costs memory per request. The common compromise, a sliding-window counter, keeps two counters, this window and the last, and weights the previous one by how much of it still overlaps. Halfway through the current minute, with 40 requests last minute and 25 so far this minute, the estimate is 25 + 40 × 0.5 = 45.
Limits usually stack: per IP address to stop floods before authentication, then per key or user for fair use, sometimes per tenant for spend.
In DocSlurp
Requests are limited per IP, then per API key or user, with sliding-window counters stored in Redis. A key can tighten its own request rate, cap pages uploaded per day, and cap spend. Responses carry the IETF RateLimit and RateLimit-Policy headers so clients can see how much room remains.
The gotcha
Rate limits count requests, not cost. One request can trigger a cheap lookup or a long model call. Pair request limits with concurrency caps and spend budgets, and with backpressure for when the whole system is busy.
In DocSlurp
In DocSlurp
Limits apply per IP first, then per API key or user, using sliding-window counters in Redis with an in-process fallback when Redis is unreachable. Keys can add their own request, page and spend windows, and responses carry RateLimit and RateLimit-Policy headers (core/server/src/middleware/rate-limit.ts).