Agents vs Workflows
The maturity question: when NOT to build an agent.
Prerequisites
- Agent Foundations — what "the model owns control flow" means in practice.
- Tool Calling — both agents and workflows call tools; ownership of the path is the difference.
The intuition
A workflow is a subway map: the tracks are laid in advance. Trains (LLM calls, tools, checks) move along known routes. Delays are annoying but the topology is inspectable — you know which station failed.
An agent is a taxi with a smart driver: you give a destination and a set of allowed roads; the driver chooses turns from traffic and signs. Great for destinations you cannot pre-route. Terrible when you needed a fixed commute budget and a testable timetable.
Most production systems need more subway than taxi. Flexibility is real value — but only where the route truly cannot be drawn ahead of time.
Key insight
Who owns control flow? If your code decides the next step at design time, it is a workflow — even if every step is an LLM. If the model decides the next step at runtime from observations, it is an agent. Tools alone do not make an agent.
Why it exists
"Build an agent" became the default pitch for any multi-step LLM feature. That default is expensive.
- Predictability. Workflows take the same path for the same input class. Agents produce a trajectory distribution. Ops, SLAs, and finance plan around the first; they fight the second.
- Testability. Stages admit unit tests and golden sets. Agents need multi-run outcome evals (pass@k, cost, steps). If you cannot fund trajectory evals, agency is under-governed by construction.
- Cost and latency. A three-step chain is three calls. Agents average ~4x chat tokens (Anthropic) with high variance — some runs 5 steps, some 50.
- Error surface. Per-step errors multiply. Short chains with deterministic gates contain failure; open loops wander, loop, or fail open unless you build the full reliability harness.
The alternatives are not "dumb vs smart." They are rungs on a ladder: single prompt → retrieval → workflow patterns → agent. Each rung costs more. The discipline is evidence before the next rung.
The core idea
The distinction that matters in practice is Anthropic's: workflows orchestrate LLMs and tools through predefined code paths — your code owns control flow; agents let the model dynamically direct its own process — the model owns control flow. The mature position: most production "agentic" systems are workflows, and should be. A workflow is predictable (same input class, same path), testable (assert on each stage), debuggable (you know where it failed), and cost-bounded (fixed number of LLM calls). An agent trades all four for flexibility, and flexibility only pays when the task genuinely can't be enumerated in advance.
Decision criteria, stated as mechanisms: (1) Open-endedness — can you write down the steps? "Summarize ticket → classify → route" is three known steps: workflow. "Fix this bug" has an unknown, input-dependent step count and branching: agent territory. (2) Verifiability — agents need a feedback signal to steer by (tests pass, API returns 200, record exists); without ground truth to observe, the loop is just expensive wandering — a workflow with a human check wins. (3) Cost of error — agents fail unpredictably and errors compound across steps; if a mistake is expensive or irreversible, either constrain the agent behind gates or take the deterministic path. Then close with the engineering rule and its reasons: use the simplest thing that works — start with a single prompt, escalate to retrieval, then to a workflow, and only to an agent when a measured gap justifies it. That's not a platitude; every rung down that ladder is cheaper per request, lower latency, easier to eval, and easier to debug — you should have evidence before paying the next rung's tax.
How it actually works
Anthropic's five workflow patterns — know them by mechanism and when each applies:
| Pattern | Mechanism | Use when |
|---|---|---|
| Prompt chaining | Fixed sequence; each call consumes the last call's output; programmatic checks ("gates") between steps | Task decomposes into known steps; trade latency for per-step accuracy (outline → check → draft → polish) |
| Routing | Classifier step picks one of several specialized downstream prompts/models | Distinct input categories handled better separately (support triage; easy→cheap model, hard→strong model) |
| Parallelization | Sectioning: independent subtasks run concurrently, results merged. Voting: same task N times, aggregate | Subtasks are independent (speed), or diverse samples raise confidence (guardrail + answer in parallel; N code reviews) |
| Orchestrator-workers | An LLM decides the subtasks at runtime, delegates, synthesizes | Subtasks aren't predictable up front (which files a change touches) — the bridge pattern: dynamic decomposition, still bounded |
| Evaluator-optimizer | Generator loop with an LLM critic scoring/feeding back until accept or max rounds | Clear eval criteria and iteration measurably helps (translation critique, code vs style rules) |
The spectrum runs prompt → chain → routing → parallelization → orchestrator-workers → agent: each rung transfers a bit more control flow from code to model. Orchestrator-workers is where runtime dynamism appears while your code still owns the loop shape; a full agent hands the model the loop itself — tools + feedback, repeat until done, with harness caps as the only rails.
Key insight
Orchestrator-workers is often mistaken for "full agency." The model chooses subtasks, but your code still owns delegate → join → synthesize → done. A true agent owns whether to continue at all after each observation.
Why workflows dominate production: evals decompose (each stage measured alone — router accuracy, extraction F1 — instead of end-to-end trajectory evals over N runs); failures localize (stage 3 broke; an agent gives you a 40-step transcript to archaeology); cost/latency are fixed and forecastable (an agent's step count is a random variable — so is your bill); per-step error rates multiply, and a workflow keeps the chain short and puts deterministic checks between links.
When the agent is genuinely right: open-ended task + verifiable environment + tolerable error cost. Coding is the flagship because the environment refutes mistakes cheaply (compiler, tests) — feedback resets the compounding-error chain. Deep research qualifies: paths unknown, sources checkable. The test to say aloud: "Could I write this as a flowchart? If yes, I build the flowchart."
Hybrids are the real endgame: production systems mix layers — deterministic pipeline for ingest/validation, a routed workflow for known categories, an agentic loop inside the one stage that's genuinely open-ended, human gates on irreversible actions. "Agent vs workflow" is a per-stage decision, not a system identity.
On the governed enterprise platform, document Q&A is usually a workflow (retrieve → ground → answer). Agent mode is reserved for multi-tool investigation with write tools gated — not for every chat turn.
The flows
| Flow | Sequence | When it applies | What breaks it |
|---|---|---|---|
| Fixed chain | Step1 → gate → Step2 → … → answer | Enumerable steps, accuracy via decomposition | Hidden branches you paper over with "just add an agent" |
| Route then specialize | Classify → one of N handlers | Known categories with different prompts/models | Overlapping classes; router accuracy never measured |
| Orchestrator-workers | Plan subtasks → fan-out → join → synthesize | Dynamic decomposition, bounded graph | Treating it as free-form agent without join contracts |
| Contained agent stage | Workflow until bottleneck → agent loop with caps → back to workflow | One open-ended stage inside a larger pipeline | Agency for 100% of traffic when 80% is enumerable |
| Escalation ladder | Prompt → RAG → workflow → agent only on measured failure | Productizing maturity over time | Jumping to agents because demos impress |
A worked example
Support intake on the governed enterprise platform (~10–15k daily users typical scale — illustrative):
Task: inbound employee message about access, payroll, or facilities.
| Design | Path | LLM calls | Eval | Cost shape |
|---|---|---|---|---|
| Naive "agent" | Free loop over 40 tools | 5–40 variable | Trajectory evals required | High variance; ~4x chat average |
| Workflow | Route intent → specialized handler → optional retrieve → answer | 2–4 fixed | Router accuracy + handler golden sets | Forecastable |
| Hybrid | Same workflow; only "unknown / multi-system incident" class enters an agent with 8 tools and cap 12 | Mostly 2–4; tail 8–12 | Stage evals + tail pass@k | Budget the tail |
Concrete routing sketch:
- Classifier (small/cheap model):
{access, payroll, facilities, incident, other}. - Access handler: fixed retrieve from IAM runbooks + answer template.
- Incident class: agent with
search_logs,get_service_owner,open_ticket(write gated). - Parallel: toxicity/PII guardrail model runs alongside classification.
The flowchart test passes for ~80% of volume. The residual 20% of true incidents pay for agency; the other 80% never should.
What goes wrong when you skip the ladder
- Demo bias — happy-path agent demos hide long-tail compounding error and p99 latency.
- No feedback signal — "draft a strategy memo" agent wanders; a chain with human critique wins.
- Irreversible tools without gates — email/customer/prod actions in an open loop.
- Unfunded evals — shipping agency without multi-run success metrics is flying blind.
- Framework complexity first — opaque graphs before a plain chain proves the need.
Common drill-downs
The natural questions one level down — criteria, pattern choice, and why restraint is an engineering skill, not conservatism for its own sake.
When would you NOT build an agent? When the steps are enumerable (flowchart test), when there's no verifiable feedback signal to steer by, when errors are costly/irreversible, when p95 latency or budget is tight, or when you can't fund trajectory-level evals. Then a workflow — or one good prompt — wins on every axis except flexibility you didn't need.
Name Anthropic's workflow patterns and pick one for a support pipeline. Chaining, routing, parallelization (sectioning/voting), orchestrator-workers, evaluator-optimizer. Support: routing — classify intent, dispatch to specialized handlers, cheap model on easy classes, strong model or human on hard ones; add parallel guardrail checks. No agent: the categories are known.
Where's the actual line between orchestrator-workers and an agent? Who owns the loop. Orchestrator-workers: the model chooses subtasks at runtime, but your code owns the shape — delegate, join, synthesize, done. Agent: the model owns continuation itself — act, observe, decide again — until it declares done or the harness stops it. Dynamic decomposition vs dynamic control flow.
Why is "start simple" treated as a senior engineering signal? Because each step up the ladder (prompt → chain → route → orchestrate → agent) multiplies cost, latency variance, eval burden, and failure surface — so taking a step requires evidence the previous rung measurably fails. Reaching for agents because they are exciting, rather than because eval data left no alternative, is how teams inherit untestable systems.
Product wants "an AI agent." How do you scope it? Extract the tasks; run each through the three criteria (open-endedness, verifiability, error cost). Usually 80% is enumerable → workflow, and one stage is genuinely open-ended → contained agent there, with tool permissions and gates scaled to blast radius. Ship the workflow first; it's also the harness the agent stage plugs into later.
Your agent demo wowed everyone but production metrics are bad. Likely story? Demos sample the happy path; production samples the distribution. Compounding per-step errors surface on long-tail inputs, non-determinism breaks repeatability, p99 latency and token spend blow up on wandering runs. Fix: carve the enumerable majority into workflow stages, keep agency for the residual, add caps/gates/evals — i.e., re-answer this page's question honestly.
Coding agents work — why doesn't that generalize to everything? Coding has what most domains lack: dense, cheap, objective feedback (compiler, types, tests) that refutes bad steps immediately and resets error compounding. Domains with slow, expensive, or subjective verification (strategy, legal judgment, taste) starve the loop of signal — there, workflows with human checkpoints remain the mature choice.
Production concerns
- Cost: a 3-step workflow is 3 calls, cacheable and budgetable. Agents average ~4x chat-interaction tokens (Anthropic's number) with high variance — some runs 5 steps, some 50. Finance wants the workflow. See cost.
- Latency: workflows parallelize deterministically and admit per-stage model right-sizing (cheap model for classification, strong for synthesis). Agent latency is step-count-distributed; p99 is ugly. See latency.
- Testing: workflow = unit tests per stage + golden datasets. Agent = statistical evals (N runs, pass@1/pass^k) — a much bigger QA investment; if you can't fund trajectory evals, that alone argues for the workflow. See evals and testing.
- Failure containment: workflows fail closed (stage errors, pipeline halts, alert fires with a stage name). Agents fail open — wandering, unproductive loops, wrong-but-confident output — demanding the whole reliability apparatus (caps, budgets, gates) before ship. See agent reliability.
- Maintenance drift: models update; a workflow re-validates stage by stage; an agent's behavior distribution shifts wholesale and needs full eval re-runs.
- Framework caution (Anthropic's, worth quoting): frameworks obscure the prompts and tempt complexity; start with direct API calls, keep prompts inspectable, adopt orchestration machinery only when checkpointing/interrupts/branching are concretely needed — many patterns are "a few lines of code." See orchestration.
- The migration path in practice: ship the workflow now; log where it's insufficient (escalations, low-confidence routes); introduce agency only at the measured bottleneck stage. Autonomy is earned by evidence, expanded gradually, never granted by default.
Test yourself
A stakeholder says every feature must be 'agentic' for marketing. How do you protect the architecture without only saying no?
Is orchestrator-workers 'basically an agent'? Defend a precise answer.
You cannot get dense automated feedback for a legal-summary task. Should you still use an agent?
Why might a 90% demo success rate still be the wrong reason to ship an agent to production?
Walk the escalation ladder for 'answer questions over our policy PDFs.'
Go deeper
- Building effective agents — Anthropic — the canonical essay this whole question is graded against: the five patterns, the definitions, "find the simplest solution."
- When to use multi-agent systems — Claude blog — the same escalation discipline one level up: single agent before many.
- Don't Build Multi-Agents — Cognition — the restraint argument from a frontier agent lab; pairs with Anthropic's essay as the two poles interviewers quote.
- How We Build Effective Agents: Barry Zhang, Anthropic — AI Engineer — the essay's co-author: don't build agents for everything; keep it simple.
- What's next for AI agentic workflows ft. Andrew Ng of AI Fund — Sequoia Capital — the popular framing of agentic patterns (reflection, tool use, planning, multi-agent) and why iteration beats one-shot.
Where this connects
- Agent foundations — the loop you only adopt when the flowchart test fails.
- Orchestration — implementing workflows and agents as graphs with checkpoints.
- Harness vs orchestration — once you've decided to build an agent, which layer each remaining problem belongs to.
- Multi-agent systems — the next escalation after a single agent; same "start simple" discipline one level up.
- Design: customer support agent — a full design where routing, tools, and escalation must be chosen deliberately.