Every AI architecture proposal landing in federal accounts right now uses the word "agent." Some of those proposals describe systems where an LLM dynamically decides what to do next based on what it finds. Most of them describe something else: a predefined code path that calls an LLM at one or more steps, with the sequence controlled entirely by the developer. Both are legitimate. Only one is an agent. The distinction matters because the governance story, the auditability posture, and the failure modes are structurally different — and your buyer's engineering lead already knows which one they're building.
This piece covers six patterns: five workflow architectures and the agent loop that sits in a different category from all of them. The taxonomy comes from Anthropic's "Building Effective Agents" framework, published in December 2024, which has since become the accepted industry reference point for this distinction. The framing was contested as recently as mid-2024; it isn't anymore. When an enterprise architect uses these terms, this is the definition they're working from.
The Six Patterns
1. Prompt Chaining
What it is: A sequence of LLM calls where the output of each step becomes the input to the next, with the sequence defined in code.
What it does: Breaks a complex task into discrete, ordered stages. A document review system might chain: extract key claims → check each claim against a policy database → generate a compliance summary. Each LLM call is scoped to one step; the code moves the output forward.
Where you'll encounter it: In proposals for document processing, multi-stage content generation, and structured data extraction. Orchestration tools like LangChain and Prefect (illustrative; accuracy review recommended) implement this as a first-class pattern. You'll also see it in vendor demos where "AI pipeline" is the preferred term.
What makes it distinct: The execution sequence is fully predetermined. The model has no say in whether step three happens — the code decides that. This is the simplest workflow pattern and the easiest to audit: you know exactly what ran, in what order, before the system touches any data.
2. Routing
What it is: An LLM (or a lightweight classifier) reads an input and directs it to one of several predefined downstream handlers.
What it does: Separates classification from execution. A citizen services chatbot might route: benefits inquiry → benefits handler, technical support → IT handler, general inquiry → general handler. The routing decision is made by the model; the handlers themselves are predefined code paths.
Where you'll encounter it: In proposals for multi-domain virtual assistants, tiered support systems, and any architecture where different input types require meaningfully different treatment. The pattern is common in contact center AI and is sometimes marketed as "intent-based routing."
What makes it distinct: The model makes exactly one decision — which path to take — and then exits. Everything downstream is deterministic. This is the pattern most often mislabeled as "agentic" in vendor pitches, because the model appears to be making a judgment call. It is, but it's a bounded one with no downstream consequences for the model.
3. Parallelization
What it is: Multiple LLM calls run simultaneously, either on different segments of the same task or on the same task multiple times for consensus.
What it does: Two sub-variants. Sectioning divides a large task into parallel chunks — processing 50 contract clauses simultaneously rather than sequentially. Voting runs the same prompt multiple times and aggregates results, typically to reduce variance on high-stakes outputs. The code controls both the division and the aggregation logic.
Where you'll encounter it: In proposals for large-document analysis, batch processing pipelines, and any system where latency is a constraint and the task is decomposable. The voting variant appears in proposals for automated decision support where accuracy requirements are high.
What makes it distinct: Parallelization is about throughput and variance reduction, not decision-making. The model never decides to parallelize — the code does. This pattern is frequently bundled with orchestrator-workers in proposals; they're related but structurally different.
4. Orchestrator-Workers
What it is: A central LLM (the orchestrator) decomposes a task and dispatches subtasks to worker LLMs, which execute and return results.
What it does: The orchestrator reads a complex task, decides how to break it into subtasks, assigns each to a worker, and synthesizes the results. A contract analysis system might have an orchestrator that identifies the relevant clauses and dispatches each to a specialized worker (liability clauses to one, IP terms to another). The orchestrator's decomposition logic is model-driven; the available workers and their capabilities are defined in code.
Where you'll encounter it: In proposals for complex document workflows, multi-domain research tasks, and enterprise automation where the task structure isn't fully predictable at design time. LangGraph and similar orchestration frameworks (illustrative) implement this pattern explicitly. This is the pattern most likely to be called "agentic" by vendors who are technically wrong but not entirely wrong — the orchestrator does exercise judgment.
What makes it distinct: The orchestrator makes real decomposition decisions, but it operates within a bounded tool set and a defined task scope. The code still defines what workers exist and what they can do. This is the closest workflow pattern to an agent loop, which is precisely why it gets mislabeled most often.
5. Evaluator-Optimizer
What it is: One LLM generates output; a second LLM evaluates it against predefined criteria; the loop continues until criteria are met or an iteration limit is reached.
What it does: Implements automated quality control within a workflow. A proposal drafting system might generate a section, evaluate it for compliance with agency style guidelines and regulatory requirements, and regenerate with feedback until the evaluation passes. The criteria are defined in code; the loop termination logic is defined in code; the model's role is generation and evaluation, not loop control.
Where you'll encounter it: In proposals for content generation, code review automation, and any workflow where output quality is variable and human review at every step is impractical. The pattern is sometimes described as "self-correcting AI" in vendor materials.
What makes it distinct: The feedback loop is real, but it's code-controlled. The evaluator LLM doesn't decide to keep looping — the code checks whether criteria were met and decides whether to continue. This pattern produces outputs that can look like they emerged from autonomous reasoning; the audit trail will show they didn't.
6. The Agent Loop
What it is: An LLM that dynamically decides which tools to call, in what order, based on what it observes at each step — with the sequence determined by the model, not the code.
What it does: The model receives a goal, selects a tool, observes the result, decides what to do next, and repeats until it determines the goal is achieved or it cannot proceed. The code provides the available tools and enforces hard stops; the model controls the execution path. A research agent given "summarize the current state of agency X's zero trust implementation" might search internal documents, query a policy database, retrieve recent audit findings, and synthesize — in an order it determines based on what each step returns.
Where you'll encounter it: In proposals for autonomous research, complex multi-step task completion, and any system where the task structure cannot be fully specified at design time. This is what vendors mean when they say "agentic" and are using the term correctly. It's also what they mean when they're using it incorrectly — which is more common.
What makes it distinct: The model is the orchestrator. There is no predefined execution path. The sequence of operations is a runtime decision made by the LLM based on intermediate results. This is the structural difference that changes everything downstream: governance, auditability, failure modes, and cost.
Comparison: Four Dimensions That Separate Them
Structure note: Six subjects across multiple dimensions don't fit a flat A/B table without losing the nuance that matters for buyer conversations. What follows is trait-led analysis — each dimension examined across all six patterns. This structure was chosen because buyers ask about dimensions ("how do you audit this?"), not about patterns in isolation.
Who Controls the Execution Path
| Pattern | Controller |
|---|---|
| Prompt chaining | Code entirely |
| Routing | Code (model makes one classification decision) |
| Parallelization | Code entirely |
| Orchestrator-workers | Code defines workers; model controls decomposition |
| Evaluator-optimizer | Code controls loop; model controls generation and evaluation |
| Agent loop | Model controls sequence; code defines tool set and hard stops |
The clean dividing line: in every workflow pattern, the code decides what happens next. In the agent loop, the model does. Orchestrator-workers is the exception that proves the rule — the orchestrator exercises judgment about decomposition, but within a code-defined boundary. That's why it's still a workflow.
What Happens When the Unexpected Occurs
Workflow patterns fail predictably. Prompt chaining breaks at the step where input doesn't match expected format. Routing misclassifies and sends the request to the wrong handler. Parallelization produces inconsistent results across workers. Orchestrator-workers hit a worker that returns an error. Evaluator-optimizer loops until it hits the iteration limit. In every case, the failure mode is bounded by the code path — you can enumerate the ways it can go wrong at design time.
The agent loop fails in ways you cannot fully enumerate at design time, because the execution path itself is dynamic. The model might call a tool in an unexpected order, interpret an intermediate result incorrectly and take a wrong turn, or pursue a plausible-looking path that leads nowhere. Production experience with agent loops has repeatedly surfaced failure modes that weren't anticipated during design — the model's decision space is genuinely larger than any predefined path, and no amount of careful engineering fully enumerates it. Agents warrant different instrumentation, not avoidance. But the instrumentation gap is real and worth naming before your buyer's security team names it first.
Auditability and Governance
Workflow patterns produce auditable traces by construction. Because the code controls the sequence, you can log every step, every input, every output, and reconstruct exactly what happened and why the system moved from step N to step N+1. For federal procurement contexts where FedRAMP, FISMA, and agency-specific audit requirements apply, that's a meaningful operational advantage. The audit story for a workflow is: here is the predefined path, here is what ran, here is what each step returned.
The agent loop requires deliberate instrumentation to produce an equivalent audit trail, and even with instrumentation, the "why" question is harder to answer. You can log what tools the model called and in what order; you cannot always reconstruct why the model made those specific decisions from the logged data alone. This is an active area of development in agent observability tooling, and it's unsettled. Buyers in regulated environments who ask "how do we audit this?" are asking a question that workflow patterns answer more cleanly than agent loops do today.
Cost and Latency
Prompt chaining and routing are the most predictable: fixed number of LLM calls, fixed sequence, predictable token consumption. Parallelization trades latency for throughput — faster wall-clock time, same or higher total token cost. Orchestrator-workers and evaluator-optimizer introduce variable call counts, but within bounded ranges that can be estimated at design time.
The agent loop has genuinely variable cost and latency, because the number of tool calls is determined at runtime by the model. An agent that decides to pursue an unexpected path may make significantly more calls than anticipated. Production deployments have reported cost variance of 3-5x between typical and edge-case executions for the same task type (this figure is illustrative; production data varies significantly by implementation). For procurement conversations where cost modeling matters, this variability is a real consideration that workflows don't share.
Field Language Guide
| Instead of... | Say... | Why it matters |
|---|---|---|
| "It's an agent" | "Is the model controlling the execution sequence, or is the code?" | Forces the vendor to name the actual architecture |
| "Agentic AI" | "Which of the workflow patterns, or is this a true agent loop?" | "Agentic" is a marketing adjective; the patterns are architectural nouns |
| "Autonomous" | "What does the model decide, and what does the code decide?" | Autonomy exists on a spectrum; the question surfaces where the boundary is |
| "Self-correcting" | "Is that evaluator-optimizer, or does the model control the loop?" | Distinguishes a workflow pattern from genuine dynamic decision-making |
| "Orchestration" | "Is the orchestrator a model or a code layer?" | "Orchestration" applies to both; the distinction changes the governance story |
| "Pipeline" | "Prompt chaining" | Pipeline is generic; prompt chaining names the specific pattern |
| "It routes intelligently" | "The routing decision is made by the model, but the downstream paths are predefined" | Prevents the classification decision from being overstated as full agency |
| "Parallel processing" | "Is that sectioning or voting?" | The two parallelization sub-variants have different use cases and different failure modes |
| "It can handle complex tasks" | "What's the task decomposition strategy — orchestrator-workers, or does the agent decide dynamically?" | Complexity is not a differentiator; the decomposition architecture is |
| "Fully automated" | "What are the hard stops, and who defines them?" | Every agent loop has code-defined limits; "fully automated" obscures where human control lives |
| "It learns from feedback" | "Is that evaluator-optimizer with predefined criteria, or something else?" | Prevents conflation with training-time learning |
| "Predictable behavior" | "Workflow patterns are predictable by construction; agent loops require instrumentation to achieve equivalent auditability" | Gives the buyer a specific, accurate claim rather than a reassurance |
Okta Concept Mapping
Workflows are lifecycle automation. Agents are something else.
The IDAM analog for workflow patterns is provisioning automation: a predefined trigger fires, a predefined sequence of operations executes, and the audit log reflects exactly what happened at each step. Joiner-mover-leaver flows, SCIM provisioning, group assignment rules — these are workflows in the identity sense, and they share the structural property that makes AI workflows governable: the code controls the sequence. The analogy holds well enough to be useful in a buyer conversation. When a CAIO asks how a workflow-based AI system fits into their governance framework, the honest answer is that it fits the same way their existing automation does — with logs, defined paths, and enumerable failure modes.
The agent loop breaks this analogy at the decision layer. A privileged service account has broad permissions but executes predefined operations; an agent loop has a defined tool set but makes real-time decisions about which tools to invoke and in what order. The closer IDAM analog is a PAM session — broad access, runtime decisions, elevated risk — except the "operator" making those decisions is the model, not a human whose judgment you've vetted through a hiring process and whose actions you can question after the fact. The governance gap that creates is real, and it's the question your buyer's security team will raise. Knowing it's coming, and knowing why it exists structurally, is what separates a fluent conversation from a reassurance that doesn't land.