Agents vs Workflows
The maturity question: when NOT to build an agent.
Prerequisites
- Agent Foundations — what "the model owns control flow" means in practice.
- Tool Calling — both agents and workflows call tools; ownership of the path is the difference.
The intuition
A workflow is a subway map: tracks laid in advance. Trains (LLM calls, tools, checks) follow known routes — when something fails, you know which station broke.
An agent is a taxi with a smart driver: destination and allowed roads, turns chosen from traffic. Great when you cannot pre-draw the route; poor when you need a fixed budget and a testable timetable. Most production systems need more subway than taxi.
Key insight
Who owns control flow? If your code decides the next step at design time, it is a workflow — even if every step is an LLM. If the model decides the next step at runtime from observations, it is an agent. Tools alone do not make an agent.
Why it exists
"Build an agent" became the default pitch for any multi-step LLM feature. That taxes four things ops funds: predictability (same path per input class vs a trajectory distribution), testability (stage unit tests vs multi-run pass@k), cost and latency (fixed call count vs Anthropic's ~4× chat tokens for agents, ~15× for multi-agent — see multi-agent systems), and error surface (per-step errors multiply; short gated chains multiply fewer of them).
The ladder is not "dumb vs smart": single prompt → retrieval → workflow patterns → agent. Each rung costs more. Move up only when the previous rung measurably fails.
The core idea
Anthropic's split is the one that holds in codebases. Workflows orchestrate LLMs and tools through predefined code paths — your code owns control flow. Agents let the model dynamically direct its own process — the model owns control flow. Both are agentic systems; the difference is architecture, not branding.
Most production "agentic" systems are workflows, and should be. Predictable path, stage-level tests, named failures, bounded cost — an agent trades all four for flexibility that only pays when the task cannot be enumerated in advance. Three criteria decide:
- Open-endedness — can you write the steps? "Summarize → classify → route" is a workflow. "Fix this bug" has input-dependent branching: agent territory.
- Verifiability — agents need a signal to steer by (tests pass, API 200). Without ground truth, prefer workflow + human check.
- Cost of error — mistakes compound. Expensive or irreversible actions need gates or a deterministic path.
Engineering rule: simplest thing that works — prompt → retrieval → workflow → agent only when metrics justify the next tax.
How it actually works
Between a single prompt and a free agent loop sit five workflow patterns Anthropic documents. Use them by mechanism, not as a checklist to ship all five.
| Pattern | Mechanism | Use when |
|---|---|---|
| Prompt chaining | Fixed sequence; each call consumes the last; programmatic gates between steps | Known subtasks; trade latency for accuracy |
| Routing | Classifier picks one specialized downstream prompt/model | Distinct categories handled better separately |
| Parallelization | Sectioning: independent subtasks concurrent. Voting: same task N times, aggregate | Speed via independence, or confidence via diversity |
| Orchestrator-workers | LLM decides subtasks at runtime, delegates, synthesizes | Subtasks unpredictable — still a bounded graph |
| Evaluator-optimizer | Generator + LLM critic until accept or max rounds | Clear criteria; iteration measurably helps |
Control flow transfers gradually: prompt → chain → route → parallel → orchestrator-workers → agent. Orchestrator-workers is the bridge: the model invents subtasks; your code still owns delegate → join → synthesize → done. A true agent owns whether to continue at all after each observation.
Key insight
Orchestrator-workers is not full agency. Dynamic decomposition is not dynamic control flow. Your code still decides that the run ends after the join.
Workflows win on decomposable evals, named stage failures, and forecastable cost. Agents win only with open-ended work, a verifiable environment, and tolerable error cost — coding is the flagship because compilers and tests refute mistakes cheaply. Spoken test: "Could I write this as a flowchart? If yes, build the flowchart."
Hybrids are the endgame: deterministic ingest, routed handlers for known classes, an agentic loop in the one open-ended stage, human gates on irreversible tools. Per stage, not system-wide.
On the governed enterprise platform, document Q&A is usually a workflow (retrieve → ground → answer). Agent mode is for multi-tool investigation with write tools gated — not every chat turn.
The flows
| Flow | Sequence | When it applies | What breaks it |
|---|---|---|---|
| Fixed chain | Step1 → gate → Step2 → … | Enumerable steps | Hidden branches papered over with an agent |
| Route then specialize | Classify → one of N handlers | Stable categories | Overlapping classes; router never measured |
| Orchestrator-workers | Plan → fan-out → join → synthesize | Dynamic decomposition, bounded graph | No join contracts; treated as free-form agent |
| Contained agent stage | Workflow → agent with caps → workflow | One open-ended bottleneck | Agency for all traffic when most is enumerable |
| Escalation ladder | Prompt → RAG → workflow → agent on failure | Maturity over time | Jumping rungs for demo optics |
A worked example
Support intake on the governed enterprise platform (~10–15k daily users — illustrative):
Task: inbound employee message about access, payroll, or facilities.
| Design | Path | LLM calls | Eval | Cost shape |
|---|---|---|---|---|
| Naive "agent" | Free loop over 40 tools | 5–40 variable | Trajectory evals required | High variance; ~4× chat average |
| Workflow | Route → specialized handler → optional retrieve → answer | 2–4 fixed | Router accuracy + handler goldens | Forecastable |
| Hybrid | Same workflow; only "incident" enters agent (8 tools, cap 12) | Mostly 2–4; tail 8–12 | Stage evals + tail pass@k | Budget the tail |
Routing sketch: cheap classifier to {access, payroll, facilities, incident, other}; access uses
fixed IAM-runbook retrieval; incident gets an agent with search_logs, get_service_owner,
open_ticket (write gated); toxicity/PII guardrail runs in parallel. Roughly 80% of volume passes
the flowchart test — only residual incidents should pay for agency.
What goes wrong when you skip the ladder
- Demo bias — happy paths hide long-tail compounding error and p99 latency.
- No feedback signal — strategy memos wander; chain + human critique wins.
- Irreversible tools without gates — email, customer, or prod actions in an open loop.
- Unfunded evals — agency without multi-run metrics is flying blind.
- Framework first — opaque graphs before a plain chain proves the need.
Production concerns
Three workflow calls are cacheable and right-sizeable (cheap model for classify, strong for synthesize). An agent's step count is a random variable — so are the bill and p99 (cost, latency). Workflows take stage unit tests plus golden sets; agents need N-run statistical evals. Cannot fund trajectory evals? Prefer the workflow (evals and testing).
Under traffic, workflows tend to fail closed (alert names the stage). Agents tend to fail open (wander, loop, wrong-but-confident) unless caps, budgets, and gates ship first (agent reliability). Model updates re-validate stage by stage for workflows; agents need a full distribution re-eval.
Anthropic's framework caution still holds: abstractions obscure prompts and invite complexity. Start with direct API calls; adopt orchestration only when checkpointing, interrupts, or branching are real needs (orchestration). Migration: ship the workflow; log escalations and low-confidence routes; add agency only at the measured bottleneck. Autonomy is earned.
Common drill-downs
How do you know the flowchart test failed without gut feel? Instrument the workflow: low-confidence routes, human escalations, stages that already "just try another tool." Agency is for residual classes that fail structurally (path unknown), not weak prompts or an untrained router.
Routing or orchestrator-workers for support? Routing when categories are stable. Orchestrator-workers when one ticket can touch an unpredictable set of systems — still with join contracts and a max-worker cap. Measure whether category coverage plateaus before paying for runtime planning.
What is a real gate between chain steps? A programmatic check, not another LLM opinion: schema validation, allow-listed next states, required citation IDs, toxicity below threshold. Fail → stop or branch.
How do you budget a hybrid tail? Separate SLOs: tight p95 for the workflow majority; higher p99 plus hard token/step/dollar caps for the agent class. Report metrics by class, not a blend that hides the tail.
Why don't coding-agent wins generalize? Coding has dense, cheap, objective feedback. Strategy, legal judgment, and taste do not — without signal, workflows with human checkpoints stay mature.
Test yourself
A stakeholder says every feature must be 'agentic' for marketing. How do you protect the architecture without only saying no?
Is orchestrator-workers 'basically an agent'? Defend a precise answer.
You cannot get dense automated feedback for a legal-summary task. Should you still use an agent?
Why might a 90% demo success rate still be the wrong reason to ship an agent?
Walk the escalation ladder for 'answer questions over our policy PDFs.'
Go deeper
- Building effective agents — Anthropic — workflows vs agents, five patterns, "simplest solution."
- How we built our multi-agent research system — Anthropic — 4× / 15× token costs; when parallel agency earns it.
- When to use multi-agent systems — Claude blog — same escalation one level up.
- Don't Build Multi-Agents — Cognition — restraint argument from a frontier lab.
- How We Build Effective Agents: Barry Zhang — AI Engineer — essay co-author: keep it simple.
- What's next for AI agentic workflows ft. Andrew Ng — Sequoia — popular "agentic" patterns; vocabulary differs from Anthropic's split.
Where this connects
- Agent foundations — the loop only when the flowchart test fails.
- Orchestration — workflows and agents as graphs with checkpoints.
- Harness vs orchestration — after you choose an agent, which layer owns which remaining problem.
- Multi-agent systems — next escalation; same start-simple discipline.
- Design: customer support agent — routing, tools, and escalation chosen deliberately end to end.