Harness vs Orchestration
Two layers that both claim the phrase "agent runtime" — where the boundary really is, and where it's just vocabulary.
Prerequisites
- The Agent Harness — the vertical layer: one loop, its tools, its context, its sandbox.
- Orchestration & Memory — the horizontal layer, as LangGraph implements it: nodes, edges, checkpointing, interrupts.
- Agents vs Workflows — the prior question of who owns control flow at all.
The intuition
Ask two experienced engineers what runs an agent and you get two confident, incompatible answers.
The LangGraph engineer says: orchestration is the runtime — nodes, edges, typed state, checkpoints. The loop is just a cycle in my graph. The Claude Code engineer says: the harness is the runtime — the loop, the tools, the sandbox, the permission gates. Orchestration is what you reach for when one agent isn't enough.
Both are describing something real. Neither is the whole picture. Think in two axes, not two camps. The harness is vertical: everything between the model and the world on a single turn — what it sees, what it can touch, what it is allowed to do. The orchestrator is horizontal: everything between one unit of work and the next — what runs after this, what state survives, who approves, what happens when the process dies at step seven.
Key insight
The harness contains non-determinism turn by turn; the orchestrator constrains it step by step. A harness makes one model turn safe and well-informed. An orchestrator makes a sequence of turns survivable, resumable, and composable. Neither substitutes for the other. Most production failures come from investing in one when the problem was in the other.
Why it exists
This looks like a vocabulary argument. It costs real money in both directions.
Orchestration bought for a harness problem. A team ships a supervisor graph with four specialist agents for "code quality." Each agent has an unnamespaced tool catalogue, no result truncation, no sandbox, and a stale instruction file. The graph is beautiful. The agents flail. Topology multiplied a per-turn failure by four and added a merge problem on top.
Harness bought for an orchestration problem. A team hand-rolls an excellent loop — good tools, tight compaction, real permission tiers — and points it at a five-hour invoice-approval process with a human sign-off in the middle. There is no durable identity for "step 7 of this approval," so a deploy mid-run loses the work, and a retry re-sends the payment.
The words hide the boundary. Vendors market their whole stack under whichever word they own. LangGraph calls itself "an agent runtime and low-level orchestration framework" — both layers, one product. Anthropic presents planner/generator/evaluator arrangements as harness design, which is orchestration-shaped work under the other word. Neither is wrong; both make the boundary invisible if you only read one.
Hold the distinction because the two layers fail differently, are tested differently, and are fixed by different people. Knowing which layer you are in tells you where to look.
The core idea
A harness is the runtime for one control loop's relationship with the model and the world: instruction and context assembly, tool schemas and dispatch, result shaping, the context budget, the sandbox, permission tiers, and the stop condition. Orchestration is the composition of multiple units of work over time: what runs next, what durable state carries between units, how branches fan out and merge, how a step retries or compensates, where a human interrupts, and how a run resumes after a crash.
The sharpest test: if it changes what the model can see or do this turn, it is harness; if it changes what happens after this unit of work finishes, it is orchestration.
The split is partly real engineering and partly dialect. The coding-agent community built vocabulary around one long-lived loop, so harness grew to mean everything — LangChain's harness anatomy even lists "orchestration logic (subagent spawning, handoffs, model routing)" as a harness component. The framework community built vocabulary around composing steps, so orchestration grew to include the per-turn tool loop inside every node. Birgitta Böckeler (on Martin Fowler's site) calls the broad definition "a very wide definition, and therefore worth narrowing down."
So use the words for concerns, not product categories. Ask "is this component doing harness work or orchestration work?" That always has an answer, even when "is this product a harness or an orchestrator?" does not.
How it actually works
Who owns which concern
Several concerns live at both layers with different semantics. Conflating them is how teams end up with two half-implementations.
| Concern | Primary layer | Also at the other layer? |
|---|---|---|
| System prompt assembly | Harness — built fresh every turn | Orchestrator picks which role a unit runs with |
| Tool schemas | Harness — the interface the model sees | Orchestrator decides which tool set a unit gets |
| Tool execution | Harness — always | No |
| Retry on tool error | Harness — return error as observation, let the model adapt | Orchestrator retries the whole unit when the unit failed |
| Context compaction | Harness — the window is a per-loop resource | Orchestrator can force a fresh unit with a handoff artefact |
| Checkpointing | Both, differently | Harness: session resume. Orchestrator: replay between units |
| Human approval | Both, differently | Harness gates a tool call; orchestrator interrupts between units |
| Branching / fan-out | Orchestrator — inspectable edges, merge semantics | Harness: parallel tool calls in one turn (not the same thing) |
| Multi-agent handoff | Orchestrator — control transfer between units | Harness spawns sub-agents as context isolation |
| Idempotency | Orchestrator's requirement, harness's implementation | Keys live on the tool; the need arises from replay |
| Observability spans | Both — and they must nest | Turn/tool spans inside unit spans inside a run |
The awkward cases
LangGraph prebuilt ReAct agent. Tool schemas, a tool node, a cycle back to the model — a harness
expressed in an orchestration framework's vocabulary. The while loop is the graph.
Claude Code sub-agents. Spawning a child with its own window looks like orchestration. Mechanically it is context management as a tool. The tell: the parent cannot resume a half-finished fan-out — no durable step identity.
OpenAI Agents SDK handoffs. The SDK defines orchestration as which agents run, in what order, split into LLM-decided and code-decided flow. Handoffs are horizontal control transfer (orchestration). The runner that executes each agent's turns is a harness in the same package.
Temporal-durable agent loop. Unambiguously orchestration: event-sourced history, deterministic replay, durable timers for human waits. It does not give you prompt assembly, tool ergonomics, or a context policy. Cleanest case — the layers barely overlap.
MCP client. Neither. MCP is plumbing into the harness's tool catalogue. Which server a unit gets is configuration at either layer.
The flows
| Flow | Shape | When it's right | What breaks it |
|---|---|---|---|
| Bare loop | while + tools, in-process | Seconds-to-a-minute, reversible actions | Must survive a deploy or wait on a human |
| Harness, no orchestrator | Rich single loop: compaction, sandbox, tiers, session resume | Interactive coding / one long conversation | Multi-hour work with irreversible steps, no step identity |
| Orchestrator, thin harness | Durable workflow of simple LLM calls | Enumerable pipelines; LLM fills stages | Open-ended work — you fight the graph for "model decides" |
| Both | Durable workflow whose units each run a full harness | Long-horizon work with approvals and side effects | Cost and complexity — earn it with real failure modes |
A worked example
On the governed enterprise platform, finance wants: "every month-end, reconcile supplier invoices against POs, flag mismatches over £1k, and post corrections after a controller signs off."
Three properties decide architecture: it runs for hours, a human sits in the middle, and the last step moves money.
| Requirement | Layer | Why |
|---|---|---|
| Read 4,000 invoices without flooding the window | Harness | Result caps, pagination, offload-and-reference |
| Model decides how to investigate each mismatch | Harness | Open path per invoice — the agent loop |
| "Reconcile invoice X" as a retryable unit | Orchestration | Durable identity so failure at 2,847 does not restart at 1 |
| Controller signs off next morning | Orchestration | Durable interrupt; do not hold a process for 14 hours |
post_correction never double-posts | Both | Orchestration creates replay risk; harness puts the idempotency key on the tool |
| Deploy a prompt fix with 400 runs in flight | Orchestration | Version in-flight runs; no harness feature does this |
Wrong layer, specific failure. One heroic loop: first mid-run deploy loses six hours of work, and a retry re-posts corrections that already landed. Pure graph, thin harness: every node blows the window on line items, the model picks the wrong of three overlapping lookup tools, and you blame the graph.
Common misconception
"We added LangGraph, so context and tools are handled." A graph gives durable state, branching, and interrupts. It does not give usable tool schemas, safe truncation, a compaction policy, or a sandbox. Every node still contains a loop that needs those. Frameworks move harness decisions into configuration; they never remove them.
Common drill-downs
Where does human-in-the-loop live? Both. Harness gates a specific tool call before execution. Orchestrator interrupts between units and parks the run for hours without holding compute. You usually need both.
Where does checkpointing live? Both. Harness: session transcript for conversation resume. Orchestrator: snapshots between units so a crash replays without re-executing side effects. LangGraph's checkpointer serves both — that is why people conflate them.
Is a sub-agent orchestration? Usually not — context isolation as a tool call. It becomes orchestration when units are independently retryable, checkpointed, and resumable after the parent dies. Ask: can I resume a half-finished fan-out?
What earns an orchestration framework? At least two of: runs that outlive a request, multi-hour human waits, side effects that must not replay, branching a loop cannot express, fleets you must version and trace. Absent those, start with direct API calls — frameworks "create extra layers of abstraction that can obscure the underlying prompts and responses" (Anthropic).
What earns harness investment instead? Failures inside a turn: wrong tool from an overlapping catalogue, results that blow the window, quality decay over a long session, unactionable errors, unsafe actions. A better graph never sees these.
Why is durable execution an agent topic now? Agents stress old workflow problems. Temporal: durable execution is "crash-proof execution" — work resumes in a new process with state intact. The non-deterministic step now costs money, branches unpredictably, and takes minutes, so replaying a decision beats re-making it.
Multi-agent slower and worse than the single agent it replaced? Usually not topology. Check shared context, parallel reads vs writes, and whether each agent's harness was tuned. Adding agents multiplies per-turn weaknesses. See multi-agent systems.
Production concerns
Instrument both layers. Spans nest: run → unit → turn → tool call. Orchestration-only cannot tell "model missed it" from a truncated tool result; harness-only cannot see unit 12 replayed twice. See observability.
Cost attribution needs the same nesting. Tokens accrue in the harness; fan-out and unit retry are orchestration decisions. Tokens-by-model without unit tags answers "what," not "why." See cost.
Idempotency is the seam that fails most. Orchestration creates replay risk; the harness owns the tool that must tolerate it. Classic symptom: duplicate write after a "safe" resume.
Deploys break in-flight harnesses. Change a prompt or tool schema mid-run and agents see a world that does not match their transcript. Version the harness; finish in-flight runs on the version they started with.
Neither layer is a security boundary alone. Harness: per-tool permissions. Orchestrator: which unit gets which capability. Private data + untrusted input + outbound channel is exploitable either way. See security.
Framework-boundary trap. Adopt orchestration, inherit its harness defaults, and you cannot explain your own agent. Understand the code under the framework.
Test yourself
Your agent keeps calling the wrong one of three similar lookup tools. Your teammate proposes a supervisor that routes to specialists. What do you say?
A run crashed at step 7 of 10 and resumed correctly — but the customer got two refund emails. Which layer failed?
Agent reads a 900-page contract, extracts obligations, waits for legal on each high-risk one. Split across the two layers.
Someone argues 'a harness IS orchestration — my LangGraph ReAct agent has nodes, edges, and state.' Steelman, then answer.
Hard latency budget: cut durable checkpointing between steps, or context compaction. Which, and what do you accept?
Go deeper
- Building effective agents — Anthropic — workflow-vs-agent split; start with direct API calls before frameworks.
- Agent orchestration — OpenAI Agents SDK — "which agents run, in what order"; LLM-decided vs code-decided flow.
- LangGraph — LangChain — "an agent runtime and low-level orchestration framework."
- The anatomy of an agent harness — LangChain — Agent = Model + Harness; orchestration logic listed as a harness component.
- Harness design for long-running application development — Anthropic — planner/generator/evaluator as harness design.
- Effective harnesses for long-running agents — Anthropic — multi-session handoffs via progress files and git.
- What is durable execution? — Temporal — "crash-proof execution."
- How coding agents work — Simon Willison — coding agent as harness: invisible prompts + tools.
- The Coming Loop — Armin Ronacher — agent loop inside; harness-level loop outside that decides if work is actually done.
- Harness engineering for coding agent users — Birgitta Böckeler — "very wide definition"; builder harness vs user outer harness.
- Don't Build Multi-Agents — Cognition — composition breaks when units miss each other's implicit decisions.
Where this connects
- The agent harness — the vertical layer in full: components, how to build one, how to tell whether a change helped.
- Orchestration & memory — the horizontal layer: state, checkpoints, interrupts, short- vs long-term memory.
- Multi-agent systems — orchestration when units are themselves agents, and the token bill.
- Agents vs workflows — whether the model should own control flow at all.
- Reliability — idempotency, replay, and staged deploys: where the two layers must agree.