Model Landscape
Buy, host, or adapt — and how to defend the choice.
Prerequisites
- How LLMs Work — enough internals to talk about what fine-tuning actually changes.
- Cost and Latency — the two axes most model decisions are really made on.
Why this section exists
"Which model should we use?" is not a leaderboard lookup. It's a question about reasoning under constraints that conflict: capability, cost, latency, data residency, licensing, and the fact that whatever you pick will be superseded within a quarter.
The durable answer has a shape, not a brand:
- State the constraint that dominates. Regulated data that can't leave the tenancy? That decides more than any benchmark. Sub-second interactive latency? Also decisive.
- Pick a tier, not a model. Frontier for hard reasoning, mid-tier for most production traffic, small/fast for classification and routing. Then name a current example, flagged as current-as-of.
- Route rather than commit. Most mature systems use two or three models behind one interface and send each request to the cheapest one that clears the quality bar.
- Say how you'd know you were wrong. An eval set and a measured comparison beats any opinion.
This is the fastest-rotting section in the playbook
Model names, context limits, and prices change monthly. Learn the decision framework here and treat every specific figure as an example that needs re-checking against provider docs before you rely on it.
The other question in this section is the one most often got backwards: fine-tuning vs RAG. The short version — RAG for knowledge, fine-tuning for behaviour. Fine-tuning is a poor place to store facts: expensive to refresh, no citations, no per-user permissions.
What's in here
| Page | The question it answers |
|---|---|
| Choosing Models | The proprietary landscape, capability vs cost vs latency, reasoning models, routing |
| Open Source & Self-Hosting | Ollama, vLLM, and the real self-host vs API tradeoff: cost, privacy, latency, ops burden |
| Fine-Tuning | When fine-tuning genuinely wins, LoRA at survival depth, and why it's not a knowledge store |
Where this connects
- Production — model choice is the biggest single lever on both the latency budget and the bill.
- RAG — the alternative to fine-tuning for knowledge, and usually the right one.
- Design: LLM Gateway — routing across models as an architecture, not a config flag.