The Agent Harness

The intuition

Picture a brilliant contractor with one strange set of limitations. They can reason about anything, but they have no hands, no memory between sentences, and no way to see the room — they can only read a note you hand them and write a note back. Worse, they will follow instructions in any note that reaches them, including one they find taped inside a cupboard.

Everything you build so they can actually renovate a house is the harness: the workbench of tools they can request, the assistant who fetches things and reads results back, the notebook that survives the shift change, the site foreman who refuses to let them knock out a load-bearing wall, and the fence around the property.

The model is the contractor. The harness is the entire job site. This section has used the word fifteen times already — caps enforced by the harness, gates at the tool boundary in the harness — without ever defining it. This page is that definition.

Why it exists

Agent foundations leaves you with a working while-loop. Everything between that loop and something you would put in front of a customer is harness, and each piece exists because of a specific property of the model.

1. The model is stateless and blind. It sees exactly one thing: the request you assemble. Someone must decide, every single turn, which instructions, which tools, which files, and how much history go into that request. That assembly step is not plumbing — it is the highest-leverage code in the system.
2. Tool calls are intents, not actions. The model emits {name, arguments}. Something has to validate it, execute it against the real world, catch the failure, and turn the result back into tokens. That something is code you own, and it is where every side effect in your product lives.
3. Context is finite and degrades before it fills. Long runs blow past the window, and quality falls well before the hard limit. Someone must compact, summarise, and offload.
4. Capability is indistinguishable from danger. A shell tool is remote code execution; a fetch tool is an untrusted-input channel. Enforcement has to sit in the code path, because a system prompt saying "always ask before deleting" is a suggestion to a probabilistic system.
5. The harness is the only deterministic half. You cannot unit-test the model. You can version, diff, and test the harness — which makes it the only surface where engineering discipline applies.

The alternative is not "no harness." It is an implicit harness scattered through your application: truncation in one file, retries in another, permissions in a prompt. That is the actual default, and it is why agent codebases rot fast.

The core idea

An agent harness is the non-model software that turns a completion API into an agent: it assembles what the model sees, runs the loop, dispatches the model's tool calls against a real environment, manages the context budget, enforces what is allowed, and decides when to stop. Anthropic's definition is the most quotable: "the system that enables a model to act as an agent: it processes inputs, orchestrates tool calls, and returns results" — and, critically, "when we evaluate 'an agent,' we're evaluating the harness and the model working together."

The vocabulary is genuinely unsettled, and it is worth knowing that rather than being surprised by it. The broad sense — the one this page uses, and the one products use — is "everything that isn't the weights": prompts, tools, sandbox, permissions, hooks, session. The narrow sense reserves harness for the execution layer only (call the model, route its tool calls, decide when to stop) and calls the behaviour-shaping layer — system prompt, tool descriptions, what carries across steps — the scaffold. Anthropic's own hosted-agent architecture splits the concept three ways: a session (the append-only log of everything that happened), a harness (the loop that calls the model and routes its tool calls to infrastructure), and a sandbox (where code actually runs). Hugging Face's glossary exists precisely because practitioners could not agree; it says so outright. Pick a sense, state it, and move on — an interviewer cares that you know the layer exists, not which dialect you speak.

The load-bearing consequence: every harness component is a hypothesis about a model weakness. Anthropic states it directly — "every component in a harness encodes an assumption about what the model can't do on its own, and those assumptions are worth stress testing." Their own example: context resets added to manage Sonnet 4.5's tendency to wrap up early near the context limit became "dead weight" on Opus 4.5, where the behaviour was simply gone. Harness code has a shelf life that model releases enforce.

How it actually works

The minimum viable harness

The famous claim — a working agent is a few hundred lines — is true, and worth internalising before you reach for a framework. Thorsten Ball's canonical walkthrough builds a file-editing agent in under 400 lines of Go, "most of which is boilerplate," with three tools (read_file, list_files, edit_file). His summary of the whole architecture: "It's an LLM, a loop, and enough tokens."

messages = [system_and_instruction_files, user_task]

for turn in range(max_turns):                       # harness cap, not a model decision
    response = model.generate(messages, tools=tool_schemas)
    messages.append(assistant(response))

    if not response.tool_calls:                     # stop: model emitted text only
        return response.text

    results = []
    for call in response.tool_calls:
        try:
            out = dispatch[call.name](call.arguments)
            results.append(tool_result(call.id, truncate(out)))
        except Exception as e:
            results.append(tool_result(call.id, str(e), is_error=True))  # observation, not a crash

    messages.append(tool_results_message(results))   # Anthropic: a user-role message

raise BudgetExceeded                                 # the loop never ends on its own

Three details in that skeleton are the whole trick. The stop condition is the absence of tool calls, not a "done" tool. Tool failures re-enter as observations rather than exceptions, because a crashed host loop removes the model's ability to self-correct. And the transcript is client-owned and resent in full every turn — the server remembers nothing.

What separates the toy from a production harness

Everything the skeleton omits. This is the actual content of the discipline.

Context management, one level down

This is the component most people underestimate, because the failure is quiet. Treat the window as a finite attention budget with diminishing returns, not a container to fill. Chroma's Context Rot report measured 18 models — including GPT-4.1, Claude 4, Gemini 2.5, and Qwen3 — and found performance is non-uniform as input grows even on deliberately trivial tasks; on LongMemEval, every model did substantially better with a focused ~300-token prompt than with the ~113k-token version containing the same relevant facts. The older Lost in the Middle result is the same geometry: accuracy is highest when the answer sits at the start or end of the context and sags in the middle, sometimes below the closed-book baseline.

The three levers, cheapest first:

• Tool-result clearing — drop the raw payloads of old tool calls, keep the fact that they happened. Nearly free, and usually the largest single win.
• Compaction — when nearing the limit, have the model write a summary that preserves decisions, constraints, and unresolved problems, discard redundant tool output, and continue from the summary plus the most recently touched files.
• Offload and reference — write large outputs to disk or a store and pass a path, so the model re-reads only what it needs. Sub-agents are the extreme version: a child may burn tens of thousands of tokens and hand back one or two thousand.

The flows

A worked example

On the governed enterprise platform, an employee asks agent mode:

"Find every purchase order over £50k that's been pending more than 30 days, work out what's blocking each one, and open a facilities ticket for the badge-reader items."

Harness configuration (illustrative): tools search_orders, get_approval_chain, lookup_user, read_file, create_ticket (tier 2, gated); instruction file with the org's PO taxonomy; container sandbox with no outbound network except the internal API gateway; 200k window; compaction at 70% utilisation; 40-turn cap.

The interesting turn is 23. Without compaction the run dies at turn 30 with a context-limit error and loses three minutes of work. With compaction the raw approval chains are gone — so if the user's next question is "what was the exact timestamp on PO-99102's L2 rejection?", the harness must re-fetch. That is the trade, and it is the right one: the summary preserved the answer to the question actually asked.

Common drill-downs

The questions that separate someone who has read about agents from someone who has operated one.

Define an agent harness in one sentence. The non-model software that turns a completion API into an agent: it assembles context, runs the loop, dispatches tool calls against a real environment, manages the context budget, enforces permissions, and owns the stop condition. Anthropic's phrasing: the system that processes inputs, orchestrates tool calls, and returns results. Add the consequence — you never evaluate a model alone, you evaluate a model/harness pairing.

What actually ends a turn? The model returning text with no tool-call blocks. Everything else is a harness-enforced stop: turn cap, token or cost budget, wall-clock timeout, or no-progress detection on repeated identical calls. Never rely on the model to terminate itself; a confused loop is the default failure, not an edge case.

Why return tool errors to the model instead of raising? Because the model's main advantage in a loop is self-correction, and an exception that kills the loop removes it. A tool_result with is_error: true and actionable text ("no such column owner_id; available: owner, owner_email") lets the model fix its own call next turn. Reserve hard failures for things the model cannot act on: auth, budget exhaustion, or the same failure N times in a row.

Your tool returns 80k tokens. What do you do? Never pass it through. Cap it at a sensible default, paginate or filter at the tool boundary, and — this is the part people skip — tell the model what was cut and how to get more, so truncation becomes a navigable interface rather than silent data loss. For genuinely large artefacts, write to the filesystem or a store and return a reference the model can re-read selectively.

When do you compact, and what do you keep? Compact on a budget threshold rather than on the hard limit, because quality degrades before the window fills. Keep decisions, constraints, and unresolved problems; keep the most recent turns verbatim so the user's "as I just said" still works; discard redundant tool payloads first, since they are the bulk. Prefer clearing old tool results over summarising — it's cheaper and lossless about what happened, only lossy about detail.

What does a sub-agent actually buy you, and what does it cost? Context isolation: the child burns its window on a search whose intermediate steps the parent never needs, and returns a distillation. That's a real architectural win for breadth-first read work. The cost is token multiplication and a coordination problem — children can't see each other's implicit decisions, which is why parallel sub-agents doing writes produce work that doesn't merge.

Where do permissions have to live, and why not in the prompt? At the tool dispatch boundary, in code, before execution. A prompt is a probabilistic request; a permission check is an invariant. Score tools on reversibility × blast radius, auto-approve reads, gate irreversible external effects, and watch your rejection rate — a gate nobody ever rejects is fatigue theatre, not a control.

How do you tell whether a harness change was an improvement? Fix the model, vary the harness, and run many trials per task, because a single run tells you nothing under non-determinism. Report pass@k (at least one of k trials succeeds — right when retries are cheap) or pass^k (all k succeed — right when consistency is the product), and say which. They diverge violently: as k rises, pass@k climbs toward 100% while pass^k collapses. Grade on outcomes — tests green, ticket exists — not on whether the trajectory matched one you liked.

Isn't the harness just going to get absorbed into the model? Partly, and that's the interesting question rather than a settled one. See the next section.

Production concerns

• Every component is a hypothesis with an expiry date. Anthropic's guidance is explicit: "every component in a harness encodes an assumption about what the model can't do on its own, and those assumptions are worth stress testing." Their context-reset machinery for Sonnet 4.5 became "dead weight" on Opus 4.5. Re-run your harness ablations on every model upgrade; deleting scaffolding is a normal outcome.
• How much the harness matters is genuinely contested, and you should know both sides. For interface choices the effect is large and cleanly measured: Stencil held 16 models fixed and changed only the edit-tool format, and saw +15 points average pass rate, with Grok Code Fast 1 going from 6.7% to 68.3%. But METR's long-horizon evaluation found the opposite for product harnesses — Claude Code beat their plain ReAct scaffold in only 50.7% of bootstrap samples on Opus 4.5, and Codex beat Triframe in 14.5% on GPT-5, concluding that neither specialised harness outperformed the default scaffolds. Reconcile them like this: harness effects are large when the bottleneck is expression — how the model states an edit, reads a result, or navigates a tool — and small when the bottleneck is raw long-horizon capability and a competent loop already exists.
• Approval fatigue is measured, not theoretical. Anthropic reports that Claude Code users approve 93% of permission prompts — at which point the prompt is a formality and the real control is whatever the user has already stopped reading. Their mitigation is a classifier, and even that lands at a 17% false-negative rate on genuinely overeager actions with a 0.4% false-positive rate on real traffic; better than skipping permissions entirely, explicitly not a substitute for review on high-stakes infrastructure. Gate narrowly so the gates that remain get read.
• The blast radius is real and documented. Anthropic's own incident list from internal use includes deleting remote git branches from a misinterpreted instruction, uploading an engineer's GitHub auth token to an internal cluster, and attempting migrations against a production database. None of these were jailbreaks — they were overeager initiative. The lesson is that sandbox scope and tool tiers are load-bearing, not defence in depth.
• Any harness with private data, untrusted content, and an outbound channel is exploitable — Simon Willison's "lethal trifecta." Tool results are an injection surface: fetched pages, issue comments, and file contents all arrive as tokens the model cannot reliably distinguish from your instructions. See security and agent reliability.
• Caching changes the cost curve but not its shape. Put the stable prefix — system prompt, instruction files, tool schemas — before the cache breakpoint and it bills at a large discount. The growing transcript after the breakpoint still bills at full rate, so caching flattens the constant and compaction flattens the growth. You need both. See cost.
• Instrument at the right granularity or you cannot debug at all. Session → turn → tool call, with model, stop reason, token counts, cache hits, latency, and permission decisions on each span. The OpenTelemetry GenAI semantic conventions give you a shared vocabulary here, though they are still evolving — instrument now, don't freeze dashboards on attribute names. See observability.

Test yourself

Go deeper

• Demystifying evals for AI agents — Anthropic — the clearest official definition of an agent harness, why you evaluate harness and model together, and pass@k vs pass^k.
• How to Build an Agent — Thorsten Ball — the canonical minimal harness: a code-editing agent in under 400 lines of Go, three tools, one loop.
• Effective context engineering for AI agents — Anthropic — context as a finite attention budget; compaction, note-taking, and sub-agent isolation as the three levers.
• Writing effective tools for AI agents — Anthropic — tool namespacing, pagination and truncation defaults, and actionable errors as harness design.
• Context Rot: How Increasing Input Tokens Impacts LLM Performance — Chroma — 18 models measured; why a focused prompt beats a long one containing the same facts.
• Scaling Managed Agents — Anthropic — the session / harness / sandbox decomposition, and why harness assumptions go stale as models improve.
• Claude Code auto mode — Anthropic — the 93% approve-everything statistic, real internal incidents, and the measured limits of a classifier gate.
• The Anatomy of an Agent Harness — LangChain — "Agent = Model + Harness," with each component derived from a specific model limitation.
• Agent glossary — Hugging Face — the honest note that these terms have no agreed definitions, plus the useful harness-vs-scaffold split.
• Agent SDK overview — Claude Docs — a productised harness enumerated as capabilities: tools, hooks, subagents, MCP, permissions, sessions.

Where this connects

• Harness vs orchestration — the layer above this one, and which concerns belong to which.
• Tool calling — the schemas and wire format the dispatch layer implements.
• Agent reliability — stop conditions, HITL gates, and sandboxing as reliability engineering rather than as components.
• Multi-agent systems — what happens when sub-agent spawning becomes the architecture rather than a context-management trick.
• Evals & testing — how to measure a harness change when the thing you changed and the thing you're measuring are both non-deterministic.