The Agent Harness
The runtime around the model — loop, tool dispatch, context, sandbox, permissions — and how to build one.
Prerequisites
- Agent Foundations — the loop this page builds a runtime around, and why the transcript is the only state.
- Tool Calling — schemas and
tool_use→tool_resultmechanics; the harness is what actually dispatches them. - Tokens & Context Windows — the finite budget that forces compaction on every long-running agent.
The intuition
Picture a brilliant contractor with one strange set of limitations. They can reason about anything, but they have no hands, no memory between sentences, and no way to see the room — they can only read a note you hand them and write a note back. Worse, they will follow instructions in any note that reaches them, including one they find taped inside a cupboard.
Everything you build so they can actually renovate a house is the harness: the workbench of tools they can request, the assistant who fetches things and reads results back, the notebook that survives the shift change, the site foreman who refuses to let them knock out a load-bearing wall, and the fence around the property.
The model is the contractor. The harness is the entire job site.
Key insight
Agent = model + harness. The model supplies judgement; the harness supplies everything else — memory, hands, eyes, and rails. When you evaluate "an agent," you are always evaluating a pairing, never a model on its own.
Why it exists
Agent foundations leaves you with a working while-loop. Everything between that loop and a product customers can trust is harness — and each piece exists because of a specific model limitation.
The model is stateless and blind. It sees exactly one thing: the request you assemble. Deciding which instructions, tools, files, and how much history go into that request every turn is not plumbing. It is the highest-leverage code in the system.
Tool calls are intents, not actions. The model emits {name, arguments}. Your code validates,
executes against the real world, catches failures, and turns results back into tokens. Every side
effect in your product lives here.
Context is finite and degrades before it fills. Quality falls well before the hard limit, so someone must compact, summarise, and offload.
Capability is indistinguishable from danger. A shell tool is remote code execution; a fetch tool is an untrusted-input channel. A prompt saying "always ask before deleting" is a suggestion to a probabilistic system — enforcement has to sit in the code path.
Finally, the harness is the only deterministic half. You cannot unit-test the model. You can version, diff, and test the harness — which makes it the only surface where engineering discipline applies.
The alternative is not "no harness." It is an implicit harness scattered through your application: truncation in one file, retries in another, permissions in a prompt. That default is why agent codebases rot fast.
The core idea
An agent harness is the non-model software that turns a completion API into an agent: it assembles what the model sees, runs the loop, dispatches tool calls against a real environment, manages the context budget, enforces what is allowed, and decides when to stop. Anthropic's definition: "the system that enables a model to act as an agent: it processes inputs, orchestrates tool calls, and returns results" — and "when we evaluate 'an agent,' we're evaluating the harness and the model working together."
The vocabulary is unsettled. This page uses the broad sense — everything that isn't the weights — matching LangChain: if you're not the model, you're the harness. The narrow sense reserves harness for the execution layer (call the model, route tools, stop) and calls the behaviour-shaping layer the scaffold. Anthropic's Managed Agents splits three ways: session (append-only log), harness (loop + routing), sandbox (where code runs). Pick a sense, state it, move on.
The load-bearing consequence: every harness component is a hypothesis about a model weakness. Context resets for Sonnet 4.5's "context anxiety" became dead weight on Opus 4.5. Harness code has a shelf life that model releases enforce.
How it actually works
The minimum viable harness
A working agent is a few hundred lines — true, and worth internalising before you reach for a
framework. Thorsten Ball's walkthrough builds a file-editing agent in under 400 lines of Go with
three tools (read_file, list_files, edit_file). His summary: "It's an LLM, a loop, and enough
tokens."
messages = [system_and_instruction_files, user_task]
for turn in range(max_turns): # harness cap, not a model decision
response = model.generate(messages, tools=tool_schemas)
messages.append(assistant(response))
if not response.tool_calls: # stop: model emitted text only
return response.text
results = []
for call in response.tool_calls:
try:
out = dispatch[call.name](call.arguments)
results.append(tool_result(call.id, truncate(out)))
except Exception as e:
results.append(tool_result(call.id, str(e), is_error=True)) # observation, not a crash
messages.append(tool_results_message(results)) # Anthropic: a user-role message
raise BudgetExceeded # the loop never ends on its ownThree details are the whole trick. The stop condition is the absence of tool calls, not a "done" tool. Tool failures re-enter as observations rather than exceptions — a crashed host loop removes the model's ability to self-correct. And the transcript is client-owned and resent in full every turn; the server remembers nothing.
What separates the toy from a production harness
Everything the skeleton omits. This is the actual content of the discipline.
| Component | What it solves | Cost if you skip it |
|---|---|---|
| Instruction assembly | Project conventions the model can't infer (CLAUDE.md / AGENTS.md, merged global → project → directory) | Stale files silently misdirect every turn |
| Tool catalogue | Selection accuracy collapses with overlap — few workflow-shaped tools, namespaced, strict schemas | Wrong tool, wasted turns, schema tokens on every request |
| Result shaping | One unbounded cat eats the window — paginate, cap, and tell the model what was cut | Silent truncation loses the answer; the model can't tell |
| Error surfacing | Opaque failures waste turns — return actionable text as tool_result | Dead loop instead of self-correction |
| Context management | Quality degrades before the hard limit — clear old tool payloads, compact, offload | Context-limit crash or quiet rot |
| Sandbox | Shell is RCE by design — jail → container → gVisor → microVM, plus egress policy | Blast radius equals host privileges |
| Permissions | Prompts cannot enforce — risk tiers and gates at the tool boundary | Overeagerness ships irreversible actions |
| Sub-agents | Deep search floods the parent — child gets its own window, returns a distillation | Token multiplication; lost intermediate detail |
| Session lifecycle | Crashes and long human waits — durable transcript, clean cancel, resume | Lost work; tool_use without tool_result rejects the API |
| Telemetry | Undebuggable without spans per turn and tool call | "It failed" with no path to root cause |
Common misconception
"Agents are 300 lines, so frameworks are pointless." The loop is 300 lines. Sandbox, context policy, permissions, and evals are where production time goes. The loop is not where your differentiation lives — that does not make the rest optional.
Context management, one level down
Treat the window as a finite attention budget with diminishing returns, not a container to fill. Chroma's Context Rot report (18 models) found non-uniform performance as input grows even on trivial tasks. On LongMemEval, every model did better with a focused ~300-token prompt than with the ~113k-token version containing the same relevant facts. Lost in the Middle is the same geometry: accuracy peaks at the start or end and sags in the middle.
Three levers, cheapest first. Tool-result clearing drops old raw payloads and keeps the fact the call happened — nearly free, usually the largest win. Compaction summarises decisions, constraints, and open problems, then continues with that summary plus recent turns kept verbatim. Offload and reference writes large outputs to disk and passes a path. Sub-agents are the extreme: tens of thousands of tokens in, one or two thousand back.
The flows
| Flow | Sequence | What breaks it |
|---|---|---|
| Plain turn | Assemble → model → dispatch → shape → append → repeat | No cap — a confused run never terminates |
| Gated write | Tiered tool → pause → human decides → dispatch or refuse-as-observation | Gating reads too trains humans to click yes |
| Compaction | Budget trips → summarise older turns → reinit with summary + recent | Summarising every turn; dropping recent verbatim history |
| Sub-agent | Child works in its own window → returns distilled findings | Parallel children doing conflicting writes |
| Interrupt / resume | Cancel → kill subprocess → mark cancelled → persist → resume | Transcript with tool_use and no matching tool_result |
A worked example
On the governed enterprise platform, an employee asks agent mode:
"Find every purchase order over £50k that's been pending more than 30 days, work out what's blocking each one, and open a facilities ticket for the badge-reader items."
Harness configuration (illustrative): tools search_orders, get_approval_chain, lookup_user,
create_ticket (tier 2, gated); org PO taxonomy in an instruction file; container sandbox with
egress only to the internal API gateway; 200k window; compaction at 70%; 40-turn cap.
| Turn | What the harness does | Context after |
|---|---|---|
| 1 | Assembles system + instruction file + tool schemas + user message | ~2.4k |
| 2–7 | Pages search_orders at 200 rows; returns 31 POs | ~14k |
| 8–22 | 31 × get_approval_chain (parallel where safe); each ~600 tokens | ~96k |
| 23 | Budget trips at 70%. Compaction collapses 31 chains into a table; raw chains discarded | ~21k |
| 24 | create_ticket is tier 2 — harness pauses for an approver | ~22k |
| — | Approver returns 3 hours later; session sleeps in durable storage | — |
| 25 | Dispatch with an idempotency key derived from the PO id | ~23k |
| 26 | Final text: 31 POs, 4 tickets opened, 2 escalations recommended | — |
Turn 23 is the trade. Without compaction the run dies at the context limit. With it, the raw chains are gone — so "exact L2 rejection timestamp on PO-99102?" needs a re-fetch. Correct trade: the summary kept the answer to the question that was asked.
What omitted stages look like in production
- No result shaping — one unbounded search returns 40k tokens; the window is half gone at turn 2.
- Permissions in the prompt — "confirm before creating tickets" loses a coin flip; 31 tickets open.
- Compaction drops recent turns — the model re-fetches what it just summarised and loops.
- Non-idempotent writes — process dies after the API call; resume opens duplicates.
- Errors raise instead of observe — a 429 kills the loop; the user sees a blank failure.
Production concerns
Every component is a hypothesis with an expiry date. Re-run harness ablations on every model upgrade; deleting scaffolding is normal. When an upgrade makes the agent worse, look at the harness first — scaffolding that fights the model's planning, aggressive compaction, tool formats the new model was trained differently on.
How much the harness matters is contested. For interface choices the effect is large: Stencil held 16 models fixed, changed only the edit-tool format, and saw about +15 points average pass rate (Grok Code Fast 1: 6.7% → 68.3%). For long-horizon product harnesses, METR found the opposite — Claude Code beat plain ReAct on Opus 4.5 in only 50.7% of bootstrap samples; Codex beat Triframe on GPT-5 in 14.5%. Effects are large when the bottleneck is expression (how the model states an edit) and small when a competent loop already exists and the bottleneck is raw capability.
Approval fatigue is measured. Claude Code users approve 93% of permission prompts. Anthropic's auto-mode classifier: 17% false negatives on overeager actions, 0.4% false positives on real traffic — better than no guardrails, not a substitute for review on high-stakes work. Gate narrowly so remaining gates get read.
Blast radius is real. Anthropic's internal incidents include deleting remote branches from a misread instruction, uploading a GitHub auth token, and attempting production migrations — overeager initiative, not jailbreaks. Sandbox scope and tool tiers are load-bearing.
The lethal trifecta — private data + untrusted content + outbound channel — is exploitable (Simon Willison). Tool results are injection surface. See security and agent reliability.
Caching flattens the constant; compaction flattens the growth. Cache the stable prefix (system prompt, instruction files, schemas); the growing transcript still bills full rate. You need both. See cost.
Instrument session → turn → tool call (model, stop reason, tokens, cache hits, latency, permission decisions). OpenTelemetry GenAI conventions still evolve — instrument now, don't freeze dashboards on names. See observability.
Common drill-downs
Define an agent harness in one sentence. Non-model software that turns a completion API into an agent: context assembly, loop, tool dispatch, context budget, permissions, stop condition. You evaluate a model/harness pairing, never a model alone.
What ends a turn? Text with no tool-call blocks. Everything else is harness-enforced (turn/cost/time caps, no-progress). A confused loop is the default failure, not an edge case.
Why return tool errors instead of raising?
Self-correction dies if the host crashes. Use is_error: true with actionable text. Hard-fail only
on auth, budget exhaustion, or the same failure N times.
Your tool returns 80k tokens. Cap, paginate, tell the model what was cut. Offload large artefacts and return a reference. Silent truncation is invisible data loss.
When do you compact, and what do you keep? On a budget threshold, not the hard limit. Keep decisions, constraints, open problems, and recent turns verbatim; clear old tool payloads first. Prefer clearing over summarising when you can.
Sub-agents: buy vs cost? Buy context isolation for deep reads. Cost: token multiplication and no shared decisions — parallel writers do not merge. Architecture-level spawning lives in multi-agent.
Where must permissions live? At the tool boundary, in code. Score reversibility × blast radius; auto-approve reads; gate irreversible external effects. A gate that never rejects is fatigue theatre.
How do you measure a harness change?
Fix the model, vary the harness, many trials. Report pass@k (any of k succeeds) or pass^k (all k
succeed) and say which — they diverge as k grows. Grade outcomes, not trajectories. See
evals & testing.
Test yourself
A teammate says: 'we don't need a harness, we're using the Agent SDK.' What's wrong with that sentence?
You change the file-edit tool from a diff format to a line-anchored format and pass rate jumps 15 points. What did you actually learn?
Quality degrades over long sessions even though you never hit the context limit. What is happening, and what do you change first?
Your harness gates every tool call. Rejections run under 1%. Is that well controlled?
A model upgrade lands and your agent gets worse. Where do you look first?
Go deeper
- Demystifying evals for AI agents — Anthropic — harness definition; evaluate pairings;
pass@kvspass^k. - How to Build an Agent — Thorsten Ball — minimal harness in under 400 lines of Go.
- Effective context engineering for AI agents — Anthropic — attention budget; compaction, notes, sub-agents.
- Writing effective tools for AI agents — Anthropic — namespacing, pagination, actionable errors.
- Context Rot — Chroma — 18 models; focused beats long with the same facts.
- Scaling Managed Agents — Anthropic — session / harness / sandbox; stale assumptions.
- Claude Code auto mode — Anthropic — 93% approve rate; incidents; classifier limits.
- The Anatomy of an Agent Harness — LangChain — Agent = Model + Harness.
- The harness problem — Stencil — 16-model edit-format ablation; +15 pts from interface alone.
- Agent glossary — Hugging Face — harness vs scaffold; unsettled vocabulary.
- Measuring Time Horizon — METR — product harnesses vs plain scaffolds on long-horizon tasks.
- Agent SDK overview — Claude Docs — productised harness capabilities.
Where this connects
- Harness vs orchestration — the layer above this one, and which concerns belong to which.
- Tool calling — the schemas and wire format the dispatch layer implements.
- Agent reliability — stop conditions, HITL gates, and sandboxing as reliability engineering rather than as components.
- Multi-agent systems — when sub-agent spawning becomes the architecture rather than a context-management trick.
- Evals & testing — measuring a harness change when both the change and the metric are non-deterministic.