CURRENT | Foundations

The Practical Piece

The Classification Decision That Makes Agent Cost Models Fragile

By Nora Kaplan— April 8, 2026

Feature image for article: The Classification Decision That Makes Agent Cost Models Fragile

Somewhere in a config file, a practitioner is deciding whether "extract pricing data from competitor pages" counts as reasoning or execution. A frontier model or a cheap model gets the job, and the cost architecture of the entire system rides on this single classification. No framework draws a formal boundary.

The two most widely adopted techniques for making agent systems affordable both produce real savings. A 90% cost reduction figure circulates in practitioner guides. The honest number is probably lower, and it depends on conditions that are harder to guarantee than they look.

The Practical Piece

The Classification Decision That Makes Agent Cost Models Fragile

By Nora Kaplan— April 8, 2026

Somewhere in a config file, a practitioner is deciding whether "extract pricing data from competitor pages" counts as reasoning or execution. A frontier model or a cheap model gets the job, and the cost architecture of the entire system rides on this single classification. No framework draws a formal boundary.

The two most widely adopted techniques for making agent systems affordable both produce real savings. A 90% cost reduction figure circulates in practitioner guides. The honest number is probably lower, and it depends on conditions that are harder to guarantee than they look.

The Argument

Where Agentic AI's 40% Failure Rate Actually Lives

By Rina Takahashi— April 8, 2026

Feature image for article: Where Agentic AI's 40% Failure Rate Actually Lives

AI captured 80% of global venture funding last quarter. Quarter-trillion dollars, one category. Meanwhile, Gartner projects that four in ten agentic AI projects won't survive to 2027. Both numbers are credible. Both are rational. And the gap between them is where actual deployments go quiet.

The usual explanations — immature governance, unclear business value — are real enough. But they share a habit of drawing attention away from something more structural, something visible in the per-task cost of running an agent before anyone even discusses scaling it. The economics were baked in early. Few teams saw them in time.

The Argument

Where Agentic AI's 40% Failure Rate Actually Lives

By Rina Takahashi— April 8, 2026

AI captured 80% of global venture funding last quarter. Quarter-trillion dollars, one category. Meanwhile, Gartner projects that four in ten agentic AI projects won't survive to 2027. Both numbers are credible. Both are rational. And the gap between them is where actual deployments go quiet.

The usual explanations — immature governance, unclear business value — are real enough. But they share a habit of drawing attention away from something more structural, something visible in the per-task cost of running an agent before anyone even discusses scaling it. The economics were baked in early. Few teams saw them in time.

Further Reading

Multi-Agent Patterns That Survive Production (and Why Most Don't)Real failure data from multi-agent deployments: cascading hallucinations, latency traps, and which patterns actually hold up.

Vendor Lock-in as Economic Liability: The Trust × Flexibility FrameworkTreats agentic AI vendor selection as a long-term cost structure commitment, not a technical preference you revisit later.

Quick links

What Agent Orchestration Frameworks Actually Commit You To

Agent Cost Optimization as Architecture, Not Afterthought

The $4.63M Price Tag on Unsecured Agents

The Cost Stack

The cost structure of agentic systems is genuinely strange. Per-token prices keep falling. Total bills keep climbing. The most expensive line item on paper often isn't the one that blows up your budget. And some of the biggest cost drivers have nothing to do with the models themselves.

We mapped six categories that enterprises actually hit when deploying agents at scale. A few are obvious. Others are the kind of thing you discover three months into production, staring at an invoice that doesn't match your projections. Together, they form a picture of where agent budgets go that looks very different from where most teams expect.

The Cost Stack

The cost structure of agentic systems is genuinely strange. Per-token prices keep falling. Total bills keep climbing. The most expensive line item on paper often isn't the one that blows up your budget. And some of the biggest cost drivers have nothing to do with the models themselves.

We mapped six categories that enterprises actually hit when deploying agents at scale. A few are obvious. Others are the kind of thing you discover three months into production, staring at an invoice that doesn't match your projections. Together, they form a picture of where agent budgets go that looks very different from where most teams expect.

Model Inference

The Model You Pick Sets the Ceiling

Inference accounts for roughly 85% of enterprise AI spend. The price spread is staggering: Claude Opus 4.5 runs $25 per million output tokens; GPT-5 nano costs $0.05 per million input. Same model, different provider? Expect 10x variance. Choosing where to run is a budget decision disguised as a technical one.

Token Volume

Falling Token Prices Hide a Rising Total Bill

Gartner puts the agentic multiplier at 5–30x more tokens per task compared to standard chat. Complex multi-agent workflows regularly blow past a million tokens on a single job. Per-token costs keep dropping. Volume is rising faster. The bill grows anyway.

Context Bloat

Context Windows Are Quietly Burning Your Tokens

Every token in the context window gets billed on every LLM call. One agent resending a 25,000-token document across five reasoning steps burns 125,000 input tokens on a single task. MCP tool metadata alone can eat 40–50% of available context. RAG and prompt caching cut this waste by 60–90%.

Orchestration Overhead

Coordination Tax Rivals the Agents' Own Costs

When agents hand off to other agents, the coordination layer becomes its own cost surface. A single edge case can trigger retry chains costing 50x the normal path. Framework choice compounds this: in benchmarks, some orchestration frameworks consumed nearly double the tokens of others running identical tasks.

Infrastructure Costs

Below the API Layer, a Whole Other Bill

Kubernetes provisioning, vector databases, memory backends, retry queues, load balancers. All separate line items, all easy to forget. Self-hosting above 100K requests per day saves 60–80% versus API pricing. Below that threshold, fixed GPU costs make self-hosting the more expensive option.

Observability Spend

Observability Bills Punish You for Actually Looking

Agent observability runs $25–$500/month at startup scale, $2K–$10K+ for enterprise volumes. The billing math gets adversarial fast: a 10-span trace with one evaluation score costs 12 billable units. Some teams have resorted to sampling 0.1% of traffic just to keep observability affordable.

Patterns That Pay

A single user request to an agent can trigger a cascade of LLM calls for planning, tool selection, execution, and verification. Five or more inference round-trips where a chat completion needs one. At scale, that multiplier becomes a budget line worth staring at.

Documented patterns for cutting these costs exist, and several carry production data or peer-reviewed evidence behind them. But each one trades savings for a specific kind of brittleness. The five cards below name what works, what it actually delivers, and exactly where it starts to crack.

Patterns That Pay

A single user request to an agent can trigger a cascade of LLM calls for planning, tool selection, execution, and verification. Five or more inference round-trips where a chat completion needs one. At scale, that multiplier becomes a budget line worth staring at.

Documented patterns for cutting these costs exist, and several carry production data or peer-reviewed evidence behind them. But each one trades savings for a specific kind of brittleness. The five cards below name what works, what it actually delivers, and exactly where it starts to crack.

Cost Pattern

Let a Frontier Model Think, Not Do

Plan-and-Execute splits the expensive reasoning from the cheap labor. A frontier model architects the strategy; distilled or smaller models carry out each step. Production deployments report 80% cost cuts, stretching toward 90% with full distillation. The catch: distilled executors fail on rare edge cases outside their training data. Precisely the cases where failure costs the most.

Token Reduction

Scope Isolation Starves Each Agent of Bloat

Monolithic agent architectures stuff every subagent with the full conversation history. Scope isolation walls off each agent's context to only what it needs. LangChain's multi-agent documentation shows a 67% token reduction per request through this boundary enforcement. The cost: each handoff between isolated agents adds 500ms–1.5s of summarization latency and risks dropping context that mattered.

Model Routing

Match the Model Tier to the Task

A frontier reasoning call can cost 190x more than a small model handling equivalent work. Routing each agent step by complexity, sending simple extraction to cheap models and reserving heavy inference for hard decisions, yields 30–70% savings across practitioner reports. The fragility: you now depend on a routing classifier that adds its own latency, and a single provider outage can break a multi-model pipeline in ways a single-model system never would.

Prompt Caching

Stop Resending the Same Tokens Every Call

Agents hammer the same system prompts, tool schemas, and few-shot examples on every invocation. Provider-level prompt caching (Anthropic discounts cached tokens by 90%) eliminates that redundancy. Practitioners report 45–80% cost reductions on cache-eligible workloads. The caveat worth remembering: when tool schemas update, stale cache hits silently degrade output quality unless you manage invalidation explicitly.

Plan Caching

Cache the Plan, Not Just the Prompt

The strongest evidence in this set comes from a peer-reviewed preprint: caching and adapting prior execution plans across five agent workloads cut costs by 50% and latency by 27%, retaining 96.6% accuracy. This goes beyond prompt-prefix caching to reuse the planner's actual reasoning. Requires a Plan-Act architecture to work, and effectiveness drops sharply on genuinely novel tasks where no prior plan applies.