DocSlurp Learn
How document search actually works
Every idea between a file landing in a bucket and a cited answer on screen, explained with pictures, small numbers you can check by hand and an honest note on where each technique breaks.
148 concepts43 illustrated explainers9 stages
On this page
Learning paths
Short reading orders for the questions people actually arrive with.
Your first RAG system
From a PDF to a cited answer, one idea at a time.
Why search misses the answer
The usual suspects when the right passage never shows up.
The math of similarity
Vectors, angles and distances, worked by hand.
Answers you can check
Grounding, citations and how to measure them.
Stage 1
Ingest & extract
Turn files, scans, slides and recordings into text you can trust, with a pointer back to where it came from.
- Document summaryA short model-written paragraph describing a whole document, generated once at ingestion and stored, often embedded, so search and chat can treat the file as a unit.In DocSlurp
- DPIDots per inch: how many pixels represent one inch of a page when it is scanned or rendered. It sets how many pixels each letter gets.In DocSlurp
- Format detectionDeciding what kind of file an upload really is, from its declared MIME type, its extension and the magic bytes at the start of the file.In DocSlurp
- Handwriting recognitionTranscribing handwritten text from an image. It is harder than printed OCR because letters join, vary between writers and follow no fixed font.In DocSlurp
- Image preprocessingCleaning a page image before OCR: rotating it upright, straightening skew, removing noise and converting to black and white so characters separate from the background.In DocSlurp
- Ingestion pipelineThe ordered stages that turn an uploaded file into searchable records: check it, detect its format, extract text, enrich it, chunk it, embed it and index it.In DocSlurp
- Keyframe extractionChoosing a small set of representative frames from a video, so a vision model can describe what was on screen without reading every frame.In DocSlurp
- Layout analysis◆Finding the regions on a page and labeling each one, such as heading, paragraph, table, figure, caption or footer, before reading the text inside them.In DocSlurp
- Markdown conversionRewriting an extracted document as Markdown, so headings, lists, tables and code survive as plain text that people, chunkers and language models can all read.In DocSlurp
- Math extractionRecovering equations from a page as structured notation, usually LaTeX or MathML, instead of a jumble of symbols in the wrong order.In DocSlurp
- Metadata extractionPulling facts about a document, such as title, author, dates, page or slide numbers and timestamps, out of the file or its content and storing them as fields.In DocSlurp
- OCR◆Optical character recognition turns pictures of text, such as scans, photos and image-only PDFs, into machine-readable characters, each with a confidence score.In DocSlurp
- Office file formatsWord, Excel and PowerPoint files. Modern ones (.docx, .xlsx, .pptx) are zip archives of XML parts; legacy ones (.doc, .xls, .ppt) are binary containers.In DocSlurp
- Page routingChoosing an extraction method separately for each page: keep the text layer, run OCR, or send the page to a vision model, based on what that page needs.In DocSlurp
- PDF text layer◆The text a PDF stores alongside its drawing instructions: positioned glyph runs plus font mappings, from which an extractor rebuilds words, spaces and order.In DocSlurp
- PII detectionFinding personal or secret data, such as ID numbers, card numbers, emails and API keys, in extracted text so it can be flagged, redacted or kept out of the index.In DocSlurp
- RasterizationRendering a page's vector drawing instructions into a grid of pixels, so OCR, vision models and previews can work from an image of the page.In DocSlurp
- Reading order◆The sequence in which a page's text blocks should be read. Columns, sidebars and captions make it differ from plain top-to-bottom, left-to-right order.In DocSlurp
- Speaker diarizationSplitting a recording into turns labeled by speaker, answering "who spoke when" without necessarily knowing who the speakers are.In DocSlurp
- Speech-to-text◆Automatic speech recognition turns recorded audio into text, ideally with timestamps for each segment or word, so every passage can point back to a moment in the recording.In DocSlurp
- Table extraction◆Recovering a table's grid of rows, columns, headers and merged cells from a document, so every value stays attached to the row and column that give it meaning.In DocSlurp
- Unicode normalizationRewriting text so characters that look the same use the same code points, for example turning the ligature "fi" into "f" + "i".In DocSlurp
- Vision-language model◆A language model that also takes images as input. For documents, it reads a page image and writes the content out as Markdown, tables and LaTeX in one step.In DocSlurp
Stage 2
Represent
Tokens, embeddings and the geometry that lets a computer say two passages mean the same thing.
- AttentionThe transformer operation that lets each token build its representation as a weighted mix of every other token, with weights computed from dot products.
- Bi-encoderA retrieval model that encodes the query and each passage separately into vectors, so passage vectors can be computed once at ingest and compared by similarity.In DocSlurp
- Byte-pair encodingA way to build a subword vocabulary by starting from single characters or bytes and repeatedly merging the most frequent adjacent pair into a new token.
- Contextual retrievalPrefixing each chunk with a short description of where it comes from before embedding and indexing it, so a chunk that says "it" still knows what "it" is.In DocSlurp
- Cosine similarity◆A similarity score between two vectors equal to the cosine of the angle between them: 1 for the same direction, 0 for perpendicular, −1 for opposite.In DocSlurp
- DimensionalityThe number of values in each embedding vector, typically a few hundred to a few thousand, which sets storage size and search cost.In DocSlurp
- Dot productMultiply two vectors position by position and add the results; the single number grows when the vectors point the same way and are long.
- Embedding◆A list of numbers a model assigns to a piece of text, image or audio, chosen so that inputs with similar meaning get vectors that point in similar directions.In DocSlurp
- Embedding space◆The high-dimensional space a particular embedding model maps inputs into, where nearness stands for similarity and each model has its own incompatible space.In DocSlurp
- Embedding versioningTracking which model and settings produced each stored vector, so that changing the embedding model re-embeds the corpus instead of mixing incompatible vectors.In DocSlurp
- Euclidean distanceThe straight-line distance between two vectors: square the difference in each dimension, add them up and take the square root.In DocSlurp
- Late interaction (ColBERT)◆A retrieval method that keeps one vector per token and scores a passage by matching each query token to its best passage token, then summing those best matches.Related technique
- Matryoshka embeddingsEmbeddings trained so that the first k values are themselves a usable, smaller embedding, letting you cut a vector short to save storage and compute.In DocSlurp
- Multimodal embeddingsEmbeddings from a model that maps more than one kind of input, such as text and page images, into the same space, so a text query can find a picture.In DocSlurp
- Sparse vectorsVectors with one slot per vocabulary term where almost every value is zero; the few nonzero weights say which words matter and how much.Related technique
- TokenThe unit of text a model reads and writes: a whole word, a piece of a word, a punctuation mark or a byte, depending on the tokenizer.
- Tokenization◆Splitting text into tokens, the subword pieces a model actually reads, each mapped to an integer ID in the model’s fixed vocabulary.
- TransformerThe neural network architecture behind modern language and embedding models: stacked layers of attention and small feed-forward networks over a sequence of tokens.
- VectorAn ordered list of numbers, read as an arrow from the origin: its direction and length can be compared with other vectors.
- Vector normalizationDividing a vector by its length so it has length 1, which makes dot product, cosine similarity and Euclidean distance agree on rankings.In DocSlurp
Stage 3
Index & store
Data structures that make a million passages searchable in milliseconds, and what they trade away.
- Approximate nearest neighbor search◆Finds vectors close to a query without measuring every stored vector, by walking or probing an index, at the cost of sometimes missing a true neighbor.In DocSlurp
- BM25◆A keyword ranking formula that scores a passage by how often it contains each query term, how rare that term is, and how long the passage is.In DocSlurp
- Chunk overlapRepeating some text from the end of one chunk at the start of the next, so a sentence near a boundary appears whole in at least one chunk.In DocSlurp
- Chunking◆Splitting extracted text into passages small enough to embed, index and hand to a model, each one becoming a unit that search can find and cite.In DocSlurp
- Full-text searchSearching the words inside documents, not just titles or fields, using an inverted index and a ranking function such as BM25.In DocSlurp
- HNSW◆A vector index made of layered neighbor graphs: sparse upper layers for long jumps, a dense bottom layer holding every vector for the final, precise steps.In DocSlurp
- Inverted index◆A map from each word to the list of documents (and often positions) that contain it, so keyword search reads short lists instead of every document.In DocSlurp
- IVF indexA vector index that clusters vectors around k-means centroids and, at query time, searches only the few clusters nearest the query.In DocSlurp
- k-nearest neighborsThe k stored vectors closest to a query under a chosen distance; in retrieval, the top-k passages a vector search returns.In DocSlurp
- LanceDBAn open-source embedded vector database built on the Lance columnar file format, able to keep its tables on local disk or in object storage such as S3.In DocSlurp
- N-gramsRuns of n consecutive words or characters, such as the word bigram "operating margin" or the character trigrams of "margin": "mar", "arg", "rgi", "gin".In DocSlurp
- PostgreSQL full-text searchPostgreSQL’s built-in keyword search: text becomes a tsvector of normalized lexemes, queries become a tsquery, a GIN index finds matches and ts_rank or ts_rank_cd orders them.In DocSlurp
- Product quantizationCompresses a vector by cutting it into sub-vectors and replacing each with the id of its nearest entry in a small learned codebook.Related technique
- Proposition indexingRewriting passages into short, self-contained factual statements with an LLM and indexing each statement on its own.In DocSlurp
- Semantic chunkingPlacing chunk boundaries where the topic changes, usually detected as a drop in embedding similarity between neighboring sentences, instead of at a fixed size.Related technique
- Stemming and lemmatizationReducing words to a shared base form before indexing, so "books", "booked" and "booking" all match the query "book".In DocSlurp
- Stop wordsVery common words such as "the", "of" and "is" that a keyword index drops because they appear almost everywhere and say little about relevance.In DocSlurp
- TF-IDFWeights a word in a document by how often it appears there (term frequency) times how rare it is across the collection (inverse document frequency).
- Vector databaseA database that stores embeddings next to their source records and answers "which rows are nearest to this vector?", usually through an ANN index plus metadata filters.In DocSlurp
- Vector quantization◆Storing vectors with fewer bits per number, for example rounding 32-bit floats to 8-bit integers, to cut memory and speed up distance math at a small accuracy cost.In DocSlurp
Stage 4
Retrieve
Turning a question into the right candidates: keywords, vectors, filters and query rewriting.
- Candidate depthHow many candidates each retriever hands to fusion and reranking, as opposed to how many results you finally return.In DocSlurp
- Conversational retrievalSearching on behalf of a chat turn whose meaning depends on earlier turns, usually by rewriting the follow-up into a standalone query first.In DocSlurp
- Hybrid search◆Runs keyword search and vector search on the same query and merges the two ranked lists, so a passage can be found by its exact words or by its meaning.In DocSlurp
- HyDE◆Asks a language model to write a fake answer to the question, embeds that fake answer, and uses it as the vector query to find real passages.In DocSlurp
- Lexical searchRetrieval that matches the words of the query against the words of each passage, after normalizing both, and ranks by how well they overlap.In DocSlurp
- Metadata filtering◆Restricting search to passages whose attributes match a condition, such as a tag, file type, page range or date, alongside the similarity ranking.In DocSlurp
- Multi-query retrievalRuns several versions or parts of one question as separate searches and merges the result lists, usually with reciprocal rank fusion.In DocSlurp
- Phrase and negation searchQuery operators that require words to appear together and in order ("jet charter") or exclude passages containing a word (-staging).In DocSlurp
- Query expansionAdding related terms or text to a query before searching, so passages that use different words for the same thing can still match.In DocSlurp
- Query rewritingUsing a language model to restate a vague, chatty or misspelled question as a clearer search query before retrieval.In DocSlurp
- Relevance floorA minimum match strength below which candidates are dropped, so a search can return nothing instead of its least bad passages.In DocSlurp
- Self-query retrievalTurning the structured parts of a natural-language question, such as a file name, page or document type, into metadata filters before searching.In DocSlurp
- Step-back promptingAsking a model for a more general version of a specific question, then searching with both the original and the general query.In DocSlurp
- Synonym expansionRewriting a keyword query so each term also matches its listed synonyms or expansions, such as PO for purchase order.In DocSlurp
- Top-kThe number of highest-ranked results a search returns, or that one stage passes to the next.In DocSlurp
- Typo toleranceMatching query words to indexed words that differ by a small number of character edits, so “recovry” still finds “recovery”.Related technique
- Vector search◆Embeds the query and every passage as vectors, then returns the passages whose vectors point most nearly in the same direction as the query’s.In DocSlurp
- Vocabulary mismatch◆The common case where the person searching and the author of the answer use different words for the same thing, so keyword matching finds nothing.
Stage 5
Rank & assemble
Fusing, reranking and trimming candidates into the context a model actually reads.
- Context assemblyTurning ranked search results into the block of evidence a model reads: which passages, in what order, with what labels and surrounding text.In DocSlurp
- Contextual compressionShrinking each retrieved passage to only the sentences that bear on the question before it goes into the prompt.In DocSlurp
- Cross-encoder◆A model that reads a query and a passage together in one pass and outputs a single relevance score for the pair.In DocSlurp
- Learning to rankTraining a model to order results from labeled examples, combining many signals such as BM25, vector similarity, freshness and clicks into one ranking.Related technique
- Lost in the middle◆Language models use information at the start and end of a long prompt more reliably than information buried in the middle.In DocSlurp
- Maximal marginal relevance◆Picks results one at a time, scoring each candidate by its relevance minus its similarity to what has already been picked, so near-duplicates stop crowding the list.In DocSlurp
- Reciprocal rank fusion◆Merges several ranked lists into one by giving each document 1 / (k + rank) from every list that returned it, ignoring raw scores.In DocSlurp
- Relevance scoreThe number a search system attaches to each result to order it. Its scale and meaning depend entirely on the method that produced it.In DocSlurp
- Reranking◆A second pass that re-scores a short list of retrieved candidates with a slower, more careful model, then keeps only the best few.In DocSlurp
- Result deduplicationRemoving results that repeat the same content, from exact copies of a chunk to the same paragraph pasted into two documents.In DocSlurp
- Score normalizationRescaling scores from different retrievers or queries onto a shared range so they can be compared, combined or thresholded.In DocSlurp
- Sentence-window retrievalIndex single sentences for precise matching, then expand each hit to a window of the sentences around it before the model reads it.Related technique
- Small-to-big retrievalSearch over small, precise chunks, then hand the model the larger section or page each match came from.In DocSlurp
Stage 6
Generate & answer
Language models, grounding, citations, structured output and the protocols that connect them.
- AbstentionA grounded system declining to answer because the retrieved evidence does not support one, instead of guessing.In DocSlurp
- Agentic RAGRAG where the model decides what to search for, reads the results and searches again until it has the evidence it needs, instead of retrieving once up front.In DocSlurp
- Citation◆A marker in a generated answer that points to the passage a claim came from, so a reader can open the source and check it.In DocSlurp
- Context window◆The maximum number of tokens a language model can handle in one call. Instructions, history, retrieved passages and the answer all share it.In DocSlurp
- GroundingConstraining a model to answer from supplied evidence, such as retrieved passages, rather than from what it memorized in training.In DocSlurp
- Hallucination◆Generated text that reads fluently but is not supported by, or contradicts, the source material or the facts.
- JSON SchemaA JSON vocabulary for describing the shape of JSON data: types, required properties, enums, patterns and ranges.In DocSlurp
- Large language modelA neural network, usually a transformer, trained on large amounts of text to predict the next token, and used to generate, summarize, extract and answer.
- Model Context ProtocolAn open protocol that lets AI clients discover and call tools, read resources and use prompts exposed by servers, over JSON-RPC.In DocSlurp
- OpenAI-compatible APIAn HTTP API that accepts and returns the same shapes as OpenAI’s Chat Completions endpoint, so existing SDKs and tools work by changing the base URL.In DocSlurp
- Prompt cachingA provider feature that stores the processed state of a repeated prompt prefix, so later requests that start with the same tokens are cheaper and faster.In DocSlurp
- Prompt injection◆An attack where text the model reads, such as a retrieved document or web page, carries instructions that try to override what the developer or user asked for.In DocSlurp
- ProvenanceThe recorded chain from an output back to its origin: which file, which page or time range, which extractor and which exact span produced it.In DocSlurp
- RAG◆Answering a question by first retrieving relevant passages from your documents, then asking a language model to write an answer from those passages, with citations.In DocSlurp
- Streaming responsesSending a model’s answer to the client piece by piece as it is generated, usually over server-sent events, instead of waiting for the whole response.In DocSlurp
- Structured output◆Asking a model to return data in a fixed machine-readable shape, usually JSON matching a schema, instead of free text.In DocSlurp
- System promptInstructions placed before the conversation that set a model’s role, rules and output format for every turn.In DocSlurp
- TemperatureA sampling setting that divides a model’s token scores before softmax: low values let the likeliest token dominate, high values flatten the choice.In DocSlurp
- Tool callingA model returning a structured request to run a named function with JSON arguments, which the application executes and feeds back as a result.In DocSlurp
Stage 7
Graphs & hierarchy
Entities, relations, communities and summary trees: structure beyond a flat list of chunks.
- Community detectionSplitting a graph into groups of nodes that connect more densely to each other than to the rest, so each group can be treated as one topic.In DocSlurp
- Entity extractionFinding the named things in text (people, systems, places, dates) and, usually, the relationships between them, as structured records.In DocSlurp
- Graph expansionAdding retrieval candidates by walking a knowledge graph outward, a fixed number of hops, from the entities a query mentions.In DocSlurp
- GraphRAG◆Retrieval-augmented generation over an extracted knowledge graph, using clustered communities and their summaries to answer questions about a whole corpus.In DocSlurp
- Knowledge graph◆A network of entities (nodes) joined by named relationships (edges), extracted from text so facts spread across documents can be connected.In DocSlurp
- Label propagationA community detection method where every node starts with its own label and repeatedly adopts the label most common among its neighbors until labels stop changing.In DocSlurp
- Map-reduce summarizationSummarizing text too long for one model call by summarizing each piece separately (map), then combining the partial summaries into one (reduce).Related technique
- RAPTOR◆A retrieval index built as a tree: chunks are clustered and summarized, the summaries are clustered and summarized again, and search runs over every level at once.In DocSlurp
Stage 8
Evaluate
Measuring retrieval and answers honestly, from recall@k to citation support.
- Answer relevanceWhether a generated answer addresses the question that was asked, regardless of whether its facts are true or supported.
- Benchmark leakageWhen knowledge of the test set seeps into the system being tested, so a score measures memory of the benchmark instead of skill.
- Faithfulness◆The share of claims in a generated answer that the retrieved passages actually support. An answer can be true and still unfaithful.
- Golden datasetA fixed set of questions with known correct answers or relevant passages, used to score a system the same way every time it changes.
- Hit rateThe fraction of queries for which at least one relevant result appears in the top k. Each query scores 1 or 0.
- LLM-as-judgeUsing a language model to grade outputs, such as whether an answer is supported or relevant, in place of a human reviewer or an exact-match rule.
- Mean reciprocal rankThe average, over queries, of 1 divided by the rank of the first relevant result, counting 0 when none is found.
- nDCG◆A ranking score that rewards highly relevant results near the top: each result’s graded gain is divided by log2(position + 1), summed, and compared with the best possible order.
- Offline and online evaluationOffline evaluation scores a system against fixed labeled data before release; online evaluation measures real users’ behavior, such as clicks or ratings, after release.In DocSlurp
- Precision@kThe fraction of the top k results that are relevant. Five results with two relevant gives precision@5 = 0.4.
- Recall@k◆The fraction of all relevant passages for a query that appear in the top k results. It measures what retrieval found, not how it ordered it.
- Relevance judgmentsLabels saying which documents or passages are relevant to each test query, and often how relevant, from which retrieval metrics are computed.
Stage 9
Operate
Cost, speed, isolation and reliability: running ingestion and search as a service.
- BackpressureA busy system pushing back on its callers, by making them wait in a bounded queue or refusing work with a retry hint, instead of accepting more than it can finish.In DocSlurp
- CachingKeeping the result of an expensive step, such as a query embedding or a list of nearest neighbors, so a repeat request can skip the work.In DocSlurp
- Document versioningTreating a re-uploaded file as a new version of the same document, so search sees only the newest copy while older versions stay readable for a while.In DocSlurp
- IdempotencyA request is idempotent when sending it twice has the same effect as sending it once; an idempotency key makes a non-idempotent call, like an upload, safe to retry.In DocSlurp
- Incremental indexingUpdating a search index by processing only what changed, such as new files, edited files or the stages a settings change affects, instead of rebuilding everything.In DocSlurp
- Latency percentiles◆Describing response times by rank instead of average: p95 is the time that 95% of requests finished within, so the slow tail stays visible.In DocSlurp
- Multi-tenancy◆Serving many customers from one deployment while keeping each one’s documents, indexes, caches, keys and spend invisible to the others.In DocSlurp
- ObservabilityInstrumenting a system with traces, metrics and logs so you can answer new questions about its behavior, such as why one search took four seconds, without shipping new code.In DocSlurp
- Pages per $1◆An ingestion cost measure: how many pages one dollar processes, computed as pages processed divided by the dollars spent processing them.In DocSlurp
- Pages per minuteAn ingestion speed measure: pages processed divided by the minutes it took, always scoped to a stage, document, run, worker or project.In DocSlurp
- Rate limitingCapping how many requests an identity, such as an IP address, user or API key, may make in a time window, and answering 429 when it goes over.In DocSlurp
- Sandboxing untrusted filesRunning parsers, converters and OCR on customer files inside an isolated environment with no secrets, no network and a kill timeout, so a malicious file cannot reach anything that matters.In DocSlurp
- ThroughputHow much work a system completes per unit of time, such as pages ingested per hour or searches served per second, as opposed to how long one item takes.In DocSlurp
- Usage meteringRecording every billable unit, such as model tokens, embedding calls, pages and compute seconds, as an event stamped with who caused it, so spend can be attributed, limited and billed.In DocSlurp
- WebhooksHTTP callbacks a service sends to your URL when something happens, such as a run finishing, so you react to events instead of polling for them.In DocSlurp
A to Z
A
- Abstention · Answering "I don’t know"
- Agentic RAG · Iterative retrieval
- Answer relevance · Answer relevancy
- Approximate nearest neighbor search · ANN
- Attention · Self-attention
B
- Backpressure
- Benchmark leakage · Data contamination
- Bi-encoder · Dual encoder
- BM25 · Okapi BM25
- Byte-pair encoding · BPE
C
- Caching
- Candidate depth · candidateDepth
- Chunk overlap
- Chunking · Text splitting
- Citation · Source attribution
- Community detection · Graph clustering
- Context assembly · Context packing
- Context window · Context length
- Contextual compression · Extractive compression
- Contextual retrieval · Contextual chunk embeddings
- Conversational retrieval · Conversational search
- Cosine similarity
- Cross-encoder
D
- Dimensionality · Embedding dimensions
- Document summary · Summary index
- Document versioning · Document versions
- Dot product · Inner product
- DPI · Dots per inch
E
- Embedding · Dense vector
- Embedding space · Vector space
- Embedding versioning · Re-embedding
- Entity extraction · Named entity recognition
- Euclidean distance · L2 distance
F
- Faithfulness · Groundedness
- Format detection · File type detection
- Full-text search · FTS
G
- Golden dataset · Golden set
- Graph expansion · Graph neighbor search
- GraphRAG · Graph RAG
- Grounding · Grounded generation
H
- Hallucination · Confabulation
- Handwriting recognition · HTR
- Hit rate · Hit@k
- HNSW · Hierarchical navigable small world
- Hybrid search · Hybrid retrieval
- HyDE · Hypothetical document embeddings
I
- Idempotency · Idempotency key
- Image preprocessing · Deskewing
- Incremental indexing
- Ingestion pipeline · Document pipeline
- Inverted index · Posting list
- IVF index · Inverted file index
K
- k-nearest neighbors · kNN
- Keyframe extraction · Frame sampling
- Knowledge graph · KG
L
- Label propagation · LPA
- LanceDB
- Large language model · LLM
- Late interaction (ColBERT) · ColBERT
- Latency percentiles · p50
- Layout analysis · Document layout analysis
- Learning to rank · LTR
- Lexical search · Keyword search
- LLM-as-judge · LLM-as-a-judge
- Lost in the middle
M
- Map-reduce summarization · Hierarchical summarization
- Markdown conversion · Document to Markdown
- Math extraction · Formula recognition
- Matryoshka embeddings · Matryoshka representation learning
- Maximal marginal relevance · MMR
- Mean reciprocal rank · MRR
- Metadata extraction
- Metadata filtering · Filtered vector search
- Model Context Protocol · MCP
- Multi-query retrieval · Query decomposition
- Multi-tenancy · Tenant isolation
- Multimodal embeddings · Image embeddings
O
- Observability
- OCR · Optical character recognition
- Office file formats · OOXML
- Offline and online evaluation · A/B testing
- OpenAI-compatible API · Chat Completions API
P
- Page routing · Per-page extraction routing
- Pages per $1 · Pages per dollar
- Pages per minute
- PDF text layer · Embedded text
- Phrase and negation search · Quoted phrases
- PII detection · Personally identifiable information
- PostgreSQL full-text search · tsvector
- Precision@k · P@k
- Product quantization · PQ
- Prompt caching · Prefix caching
- Prompt injection · Indirect prompt injection
- Proposition indexing · Dense X retrieval
- Provenance · Lineage
Q
- Query expansion · Query augmentation
- Query rewriting · Query reformulation
R
- RAG · Retrieval-augmented generation
- RAPTOR · Recursive abstractive processing for tree-organized retrieval
- Rasterization · Page rendering
- Rate limiting
- Reading order
- Recall@k · Recall at k
- Reciprocal rank fusion · RRF
- Relevance floor · minScore
- Relevance judgments · Qrels
- Relevance score
- Reranking · Re-ranking
- Result deduplication · Near-duplicate removal
S
- Sandboxing untrusted files · Parser sandbox
- Score normalization · Min-max normalization
- Self-query retrieval · Self-querying
- Semantic chunking
- Sentence-window retrieval
- Small-to-big retrieval · Parent document retrieval
- Sparse vectors · Learned sparse retrieval
- Speaker diarization · Who spoke when
- Speech-to-text · ASR
- Stemming and lemmatization · Lemmatization
- Step-back prompting · Step-back query
- Stop words
- Streaming responses · Server-sent events
- Structured output · Structured extraction
- Synonym expansion · Glossary expansion
- System prompt · System message
T
- Table extraction
- Temperature · Sampling temperature
- TF-IDF · Term frequency–inverse document frequency
- Throughput
- Token
- Tokenization · Tokenizer
- Tool calling · Function calling
- Top-k · limit
- Transformer
- Typo tolerance · Fuzzy matching
V
- Vector
- Vector database · Vector store
- Vector normalization · L2 normalization
- Vector quantization · Scalar quantization
- Vector search · Semantic search
- Vision-language model · VLM
- Vocabulary mismatch · The vocabulary problem
No concept matches that yet. Try a broader word, or search all docs.