Skip to content

LearnIngest & extract

OCR

Optical character recognition

Optical character recognition turns pictures of text, such as scans, photos and image-only PDFs, into machine-readable characters, each with a confidence score.

In DocSlurpIllustrated explainer

On this page
OCR reads a scanned line character by character, with a confidence for eachA rough scanned strip reads "Boarding at 15 min". Below it, the recognized characters each have a confidence bar. Almost all are 0.97 to 0.99. The "1" in "15" is only 0.55, because the recognizer also considered "l" at 0.38 and "I" at 0.07. The transcript reads "Boarding at 15 min", and word confidences, taken as the weakest character, are Boarding 0.97, at 0.99, 15 0.55 and min 0.98.1 · PIXELS2 · BEST GUESS3 · CONFIDENCEBoardingat5min1the recognizer reads the whole line and scores every candidate characterBoardingat15min.99.99.98.97.99.99.99.99.99.99.55.98.98.99.99TRANSCRIPTBoarding at 15 minword confidence = its weakest characterBoarding .97 · at .99 · 15 .55 · min .98CANDIDATES FOR THAT GLYPH10.55l0.38I0.07
OCR returns a guess and a confidence for every character, and the confidence is where the errors hide. Every glyph in this scanned line reads at 0.97 or better except the narrow "1", which the recognizer splits between 1, l and I. The transcript looks clean either way; only the 0.55 tells you that a search for "15 min" may not match this page. Confidences are illustrative.

The problem: a page that is only a picture

The executive assistant photographs a signed charter confirmation and uploads it as charter-confirmation.jpg. To a person it says “Boarding at 15 min”. To a computer it is a grid of 12 million colored dots. Search for “boarding” and nothing comes back, because there is no text to match.

Scanned PDFs have the same problem. Many have no text layer at all, and some carry a hidden one produced by whatever scanner software made them years ago. Before anything can be chunked, embedded or cited, someone has to turn those dots back into characters. That is optical character recognition.

How OCR turns pixels into characters

A modern OCR engine such as Tesseract works in four steps.

  1. Clean the image. Convert to black and white, straighten skew, remove specks. See image preprocessing.
  2. Find the text. Locate blocks, then lines inside each block. This is a small piece of layout analysis.
  3. Recognize each line. Since version 4, Tesseract feeds a whole line image to an LSTM neural network that slides across it and, at each step, outputs a probability for every character it knows. Decoding picks the most likely sequence.
  4. Report the result. Text, plus a bounding box and a confidence score for every word.

The figure shows step 3 on one line of the confirmation. Most characters come back at 0.97 or higher. One does not: the network is torn between “1” (0.55), “l” (0.38) and “I” (0.07). These numbers are illustrative, but the pattern is real. Narrow strokes like 1, l, I and | look alike in a blurry photo.

Character Best guess Runner-up Word
B, o, a, r, d, i, n, g ≥ 0.97 Boarding 0.97
1 1 (0.55) l (0.38) 15 at 0.55
5 5 (0.98) S (0.02)

A common convention reports a word’s confidence as its weakest character, so “15” scores 0.55 even though the “5” is nearly certain.

Confidence is the useful part

A transcript hides its own doubt. “Boarding at 15 min” and “Boarding at l5 min” look equally finished on screen. Only the confidence scores tell them apart, which makes them the most useful output OCR gives you:

  • Flag, don’t trust. Words under a threshold (say 0.6) can be sent to a stronger reader, shown to a reviewer, or kept out of exact-match fields.
  • Digits are the worst case. For words, a language model can rescue a bad glyph: “Boardlng” is obviously “Boarding”. For numbers and identifiers, nothing in the context says whether “15” or “l5” is right. Invoice totals, dates and part numbers get no help from a dictionary.
  • Search inherits the errors. If the index holds “l5 min”, a keyword search for “15 min” misses the page, and the embedding of the passage drifts too.

Where it breaks

  • Too few pixels. OCR can only read what the pixels hold. A page scanned at low DPI gives each letter a handful of pixels, and accuracy falls off quickly. Tesseract’s documentation recommends scans of at least 300 DPI.
  • Rotation and perspective. A phone photo taken at an angle, or a page scanned sideways, breaks line finding before recognition even starts.
  • The wrong language data. Tesseract recognizes only scripts it has models for. Run it with English models on a Chinese page and it does not fail loudly: it returns confident-looking runs of Latin letters.
  • Layout. OCR reads lines. It does not know that two lines belong to different columns, or that a row of numbers is a table. Reading order and table extraction are separate problems.
  • Handwriting. Engines trained on print do poorly on cursive. See handwriting recognition.
  • Old hidden layers. A scanned PDF may already contain invisible OCR text from years ago, of unknown quality. Treating it as a trustworthy PDF text layer imports someone else’s mistakes.

OCR or a vision model?

A vision-language model can also read a page image, and it handles tables, charts and messy layouts that line-based OCR cannot. The trade is real:

OCR engine Vision-language model
Runs locally, on CPU usually hosted, on GPU
Output text, boxes, per-word confidence text, usually no confidence
Failure mode garbled characters you can see fluent text that may be invented
Layout lines only tables, columns, math

A sensible pipeline uses both: the text layer when it is trustworthy, OCR for plain scans, and a vision model for the pages that need it. That decision is page routing.

In DocSlurp

In DocSlurp

PaddleOCR PP-OCRv6 (with per-script recognisers for Korean, Cyrillic, Arabic, Devanagari, Greek and Thai) reads images and scanned pages inside an isolated VM, one per job, never in the worker process (core/worker/src/ocr/, core/docs/extraction/ocr.md). Pages it reads poorly, and Arabic or Hindi pages, get a second read from a hosted vision model, within a per-document budget. The fast quality mode never runs OCR.

Sources

Jump to