Skip to content

LearnIngest & extract

Markdown conversion

Document to Markdown

Rewriting an extracted document as Markdown, so headings, lists, tables and code survive as plain text that people, chunkers and language models can all read.

In DocSlurpGlossary entry

On this page

Markdown conversion rewrites an extracted document as Markdown. A PDF, a Word file or a web page becomes plain text where # marks a heading, - a list item, | a table row and triple backticks a code block.

Markdown is the common target for document pipelines because it sits in the middle:

  • Structure survives. A heading in the source becomes ## Margin plan, so a chunking step can split at section boundaries and record the heading path for each chunk.
  • It stays plain text. Language models read Markdown fluently and cite it naturally. People can read it too.
  • It is lossy in the right places. Fonts, colors and exact positions are dropped; they rarely matter for search.

A page of growth-plan-2027.pdf might come out as:

## 4.2 Margin plan

Operating margin reaches 24% in FY2027, four points above the 20% target.

| Metric | Target | FY2027 |
|---|---|---|
| Operating margin | 20% | 24% |

The gotcha: plain Markdown can’t express everything. Pipe tables have no merged cells, so a table with spans is either flattened (often wrongly, see table extraction) or written as an embedded HTML table, which Markdown allows. Math needs a convention such as LaTeX between $ signs. Pick the representation that keeps the meaning, even if it is less pretty; fidelity matters more than a pure Markdown look.

In DocSlurp

In DocSlurp

Every format becomes page-by-page Markdown. PDFs go to the sandboxed VM extractor where one is configured, otherwise the native PDF engine, then poppler text; Word, OpenDocument, RTF, EPUB, HTML, CSV and text go to pandoc, PowerPoint and Excel to python-pptx and openpyxl sidecars (core/worker/src/extractors/registry.ts). When tables must be preserved, Markdown tables are converted to HTML tables, which can express merged cells (core/worker/src/lib/extracted-markdown.ts).

Sources

Jump to