Skip to content

LearnGenerate & answer

Large language model

LLM

A neural network, usually a transformer, trained on large amounts of text to predict the next token, and used to generate, summarize, extract and answer.

Background conceptGlossary entry

On this page

A large language model (LLM) is a neural network trained to predict the next token of text, given the tokens before it. Trained on enough text, that one skill turns into many: answering questions, summarizing, translating, extracting fields, writing code. Most current LLMs are transformers.

Generation is a loop. The model reads the prompt, produces a probability for every token in its vocabulary, one token is picked (see temperature), appended, and the loop runs again. “FY2027 revenue is projected to” might continue with “grow” because that word is likely, not because the model checked your growth plan.

That is the key to using LLMs well. A model knows what text usually looks like. It does not know your documents, anything after its training cutoff, or whether a fluent sentence is true. Three habits follow:

  • Supply the facts. Put the relevant passages in the prompt instead of hoping the model memorized them. That is RAG.
  • Ask for pointers. Have the model cite which passage each claim came from, so a person can check it. See citation.
  • Budget the input. Everything the model reads and writes must fit its context window, and providers bill per token in both directions.

The gotcha

Fluency is not evidence. An LLM produces confident, well-formed text whether or not it has the facts, and its tone doesn’t change when it is wrong. Treat its output as a draft to verify, which is why grounded systems pair every answer with sources. See hallucination.

In DocSlurp

Background concept

DocSlurp calls hosted models through OpenRouter, or an operator-registered private endpoint (llm://private/...) when a project privacy mode forbids hosted calls (core/worker/src/lib/ai.ts, core/server/src/lib/privacy.ts). Every call records its token usage.

Sources

Jump to