OCR
Optical character recognition
Optical character recognition turns pictures of text, such as scans, photos and image-only PDFs, into machine-readable characters, each with a confidence score.
In DocSlurpIllustrated explainer
On this page
The problem: a page that is only a picture
The executive assistant photographs a signed charter confirmation and uploads it as charter-confirmation.jpg. To a person it says “Boarding at 15 min”. To a computer it is a grid of 12 million colored dots. Search for “boarding” and nothing comes back, because there is no text to match.
Scanned PDFs have the same problem. Many have no text layer at all, and some carry a hidden one produced by whatever scanner software made them years ago. Before anything can be chunked, embedded or cited, someone has to turn those dots back into characters. That is optical character recognition.
How OCR turns pixels into characters
A modern OCR engine such as Tesseract works in four steps.
- Clean the image. Convert to black and white, straighten skew, remove specks. See image preprocessing.
- Find the text. Locate blocks, then lines inside each block. This is a small piece of layout analysis.
- Recognize each line. Since version 4, Tesseract feeds a whole line image to an LSTM neural network that slides across it and, at each step, outputs a probability for every character it knows. Decoding picks the most likely sequence.
- Report the result. Text, plus a bounding box and a confidence score for every word.
The figure shows step 3 on one line of the confirmation. Most characters come back at 0.97 or higher. One does not: the network is torn between “1” (0.55), “l” (0.38) and “I” (0.07). These numbers are illustrative, but the pattern is real. Narrow strokes like 1, l, I and | look alike in a blurry photo.
| Character | Best guess | Runner-up | Word |
|---|---|---|---|
| B, o, a, r, d, i, n, g | ≥ 0.97 | Boarding 0.97 | |
| 1 | 1 (0.55) | l (0.38) | 15 at 0.55 |
| 5 | 5 (0.98) | S (0.02) |
A common convention reports a word’s confidence as its weakest character, so “15” scores 0.55 even though the “5” is nearly certain.
Confidence is the useful part
A transcript hides its own doubt. “Boarding at 15 min” and “Boarding at l5 min” look equally finished on screen. Only the confidence scores tell them apart, which makes them the most useful output OCR gives you:
- Flag, don’t trust. Words under a threshold (say 0.6) can be sent to a stronger reader, shown to a reviewer, or kept out of exact-match fields.
- Digits are the worst case. For words, a language model can rescue a bad glyph: “Boardlng” is obviously “Boarding”. For numbers and identifiers, nothing in the context says whether “15” or “l5” is right. Invoice totals, dates and part numbers get no help from a dictionary.
- Search inherits the errors. If the index holds “l5 min”, a keyword search for “15 min” misses the page, and the embedding of the passage drifts too.
Where it breaks
- Too few pixels. OCR can only read what the pixels hold. A page scanned at low DPI gives each letter a handful of pixels, and accuracy falls off quickly. Tesseract’s documentation recommends scans of at least 300 DPI.
- Rotation and perspective. A phone photo taken at an angle, or a page scanned sideways, breaks line finding before recognition even starts.
- The wrong language data. Tesseract recognizes only scripts it has models for. Run it with English models on a Chinese page and it does not fail loudly: it returns confident-looking runs of Latin letters.
- Layout. OCR reads lines. It does not know that two lines belong to different columns, or that a row of numbers is a table. Reading order and table extraction are separate problems.
- Handwriting. Engines trained on print do poorly on cursive. See handwriting recognition.
- Old hidden layers. A scanned PDF may already contain invisible OCR text from years ago, of unknown quality. Treating it as a trustworthy PDF text layer imports someone else’s mistakes.
OCR or a vision model?
A vision-language model can also read a page image, and it handles tables, charts and messy layouts that line-based OCR cannot. The trade is real:
| OCR engine | Vision-language model | |
|---|---|---|
| Runs | locally, on CPU | usually hosted, on GPU |
| Output | text, boxes, per-word confidence | text, usually no confidence |
| Failure mode | garbled characters you can see | fluent text that may be invented |
| Layout | lines only | tables, columns, math |
A sensible pipeline uses both: the text layer when it is trustworthy, OCR for plain scans, and a vision model for the pages that need it. That decision is page routing.
In DocSlurp
In DocSlurp
PaddleOCR PP-OCRv6 (with per-script recognisers for Korean, Cyrillic, Arabic, Devanagari, Greek and Thai) reads images and scanned pages inside an isolated VM, one per job, never in the worker process (core/worker/src/ocr/, core/docs/extraction/ocr.md). Pages it reads poorly, and Arabic or Hindi pages, get a second read from a hosted vision model, within a per-document budget. The fast quality mode never runs OCR.