Skip to content

DocSlurp Learn

How document search actually works

Every idea between a file landing in a bucket and a cited answer on screen, explained with pictures, small numbers you can check by hand and an honest note on where each technique breaks.

148 concepts43 illustrated explainers9 stages

On this page
The path from a file to a cited answerSix stages left to right: ingest and extract a page, represent it as vectors, index them, retrieve candidates for a question, rank and assemble context, then generate an answer with a page citation for the person who asked. Graphs and hierarchy branch above the index; evaluation and operations run underneath every stage.Graphs & hierarchy8 concepts1Ingest & extract23 concepts2Represent20 concepts3Index & store20 concepts4Retrieve18 concepts5Rank & assemble13 concepts6Generate & answer19 concepts“When does the Londonoffice open?”Evaluate12 concepts · is the answer right, and is it supported?Operate15 concepts · how fast, how cheap, how isolated?
One question, end to end. A file becomes located text, then vectors in an index. A question pulls candidates, ranking trims them to context, and the model answers with a page you can open. Select a stage to jump to its concepts.

Learning paths

Short reading orders for the questions people actually arrive with.

  1. Your first RAG system

    From a PDF to a cited answer, one idea at a time.

    1. Chunking
    2. Embedding
    3. Cosine similarity
    4. Vector search
    5. Hybrid search
    6. Reranking
    7. Context window
    8. RAG
    9. Citation
  2. Why search misses the answer

    The usual suspects when the right passage never shows up.

    1. Vocabulary mismatch
    2. BM25
    3. HyDE
    4. Approximate nearest neighbor search
    5. Reciprocal rank fusion
    6. Lost in the middle
    7. Recall@k
  3. The math of similarity

    Vectors, angles and distances, worked by hand.

    1. Vector
    2. Dot product
    3. Cosine similarity
    4. Euclidean distance
    5. Vector normalization
    6. Vector quantization
    7. HNSW
  4. Answers you can check

    Grounding, citations and how to measure them.

    1. Grounding
    2. Hallucination
    3. Provenance
    4. Citation
    5. Abstention
    6. Faithfulness
    7. Prompt injection

Stage 1

Ingest & extract

Turn files, scans, slides and recordings into text you can trust, with a pointer back to where it came from.

Stage 2

Represent

Tokens, embeddings and the geometry that lets a computer say two passages mean the same thing.

Stage 3

Index & store

Data structures that make a million passages searchable in milliseconds, and what they trade away.

Stage 4

Retrieve

Turning a question into the right candidates: keywords, vectors, filters and query rewriting.

Stage 5

Rank & assemble

Fusing, reranking and trimming candidates into the context a model actually reads.

Stage 6

Generate & answer

Language models, grounding, citations, structured output and the protocols that connect them.

Stage 7

Graphs & hierarchy

Entities, relations, communities and summary trees: structure beyond a flat list of chunks.

Stage 8

Evaluate

Measuring retrieval and answers honestly, from recall@k to citation support.

Stage 9

Operate

Cost, speed, isolation and reliability: running ingestion and search as a service.

A to Z

ABCDEFGHIJKLMNOPQRSTUVW

A

B

C

D

E

F

G

H

I

M

N

  • N-grams · Bigrams
  • nDCG · Normalized discounted cumulative gain

O

P

Q

R

S

T

V

Jump to