LLM Fundamentals
The floor everything else stands on: what the model actually is, what it costs, what it can't do.
Prerequisites
None. This is the entry point — start here if you're starting anywhere.
Why this section exists
Every system in this playbook is a wrapper around one primitive: a function that takes a sequence of tokens and returns a probability distribution over the next token. That's it. There is no memory, no database, no reasoning engine underneath — just that function, called in a loop.
Almost every surprising behaviour of LLM applications falls out of that one fact:
- The model has no memory between calls → you resend everything → context is a budget.
- It always produces a next token, however unsupported → hallucination is the default, not a bug.
- It samples from a distribution → identical inputs give different outputs, and "temperature 0" doesn't fully fix that.
- It bills by token in and token out → your architecture diagram is also your invoice.
It's worth getting this floor solid, because an engineer who has only used the API at the surface will confidently build on a false model of what happens underneath — and the resulting bugs are the confusing kind. Survival depth here is worth more than a shallow tour of a dozen frameworks.
What's in here
| Page | The question it answers |
|---|---|
| How LLMs Work | What is actually happening on each call — transformers and attention, at exactly the depth an application engineer needs |
| Tokens & Context Windows | Why text becomes tokens, what the context window really constrains, and why long context isn't free |
| Sampling & Determinism | Temperature, top-p, top-k — and why temperature=0 still doesn't guarantee identical output |
| Prompt Engineering | System prompts, few-shot, chain-of-thought, and the anti-patterns that waste tokens |
| Structured Output | Getting reliable JSON out: JSON mode vs constrained decoding vs tool-calling |
| LLM APIs | The request/response contract, streaming, prompt caching, batch APIs |
| Hallucination | Why it happens mechanically, and the layered mitigations that actually reduce it |
Read them in that order — each one uses vocabulary the previous one introduced.
Where this connects
- Embeddings & Search — the other model family you'll use constantly, trained for representation rather than generation.
- RAG — the standard cure for the knowledge-cutoff and hallucination problems named here.
- Agents — what happens when you put the next-token function in a loop and give it tools.
- Production — latency, cost, and reliability all trace back to the token mechanics on these pages.