Format detection
File type detection · MIME sniffing
Deciding what kind of file an upload really is, from its declared MIME type, its extension and the magic bytes at the start of the file.
In DocSlurpGlossary entry
On this page
Format detection decides what kind of file an upload really is, so it can be sent to the right extractor. There are three sources of evidence, in increasing order of trust:
- The declared MIME type, sent by the browser or client:
application/pdf. - The extension:
.pdf. - The magic bytes: the first few bytes of the file, which most formats fix by specification.
| Starts with | Format |
|---|---|
%PDF- |
|
PK (50 4B 03 04) |
zip, which includes .docx, .xlsx, .pptx and .epub |
89 PNG |
PNG image |
| FF D8 FF | JPEG image |
RIFF … WAVE |
WAV audio |
The first two are claims made by whoever uploaded the file. The third is evidence. When they disagree, believe the bytes. A file named charter-agreement.pdf that starts with PK is a zip archive, possibly an Office document renamed by accident, possibly something crafted to reach a parser it shouldn’t.
Zip-based formats need one more step: a .docx and a .xlsx have identical magic bytes, so the detector looks inside the archive for word/ or xl/ entries.
The gotcha: a mismatch is a security signal, not just a routing problem. Parsers for complex formats are a classic attack surface, and an attacker who controls the declared type can try to steer a file into the weakest one. Detect from bytes, refuse contradictions rather than guessing, and run every parser in a sandbox. See sandboxing and Office file formats.
In DocSlurp
In DocSlurp
Two steps. The quarantine stage sniffs magic bytes (%PDF-, PK, PNG, JPEG, RIFF and more) and quarantines a file whose declared type contradicts its bytes (core/worker/src/lib/quarantine.ts). Then format-detect maps the MIME type, falling back to the extension, onto one of 16 normalized formats (including unknown) that choose the extractor (core/worker/src/stages/format-detect.ts).