Somebody shows you a benchmark. Model A: 74%. Model B: 68%. You pick A. And you have just decided based on a number that doesn't describe what you think it describes, because the result was never a property of the model. It was a property of the model plus everything around it while the test ran.
I've spent twenty-odd years watching people mistake a measurement for an understanding. It's the oldest failure mode in production work: the dashboard says 99.9%, you believe the dashboard, and then the system does something the number had no way of capturing, because the instrumentation was shaped by conditions that stopped holding weeks ago. Agent benchmarks have the same defect, and three recent studies make it hard to ignore.
ERPBench ran six model families through a hundred simulated enterprise problems — pricing, procurement, inventory — under two conditions. First, each model competed against scripted rule-based opponents. Then the models competed against each other in a shared market, where one agent's pricing decision moved the demand another agent was facing. The overall leader changed. Across those hundred matched problems, the two environments agreed on the top-scoring model 21 times. The rank correlation between them was 0.182, which means that if you had ranked the models by coin flip you'd have done nearly as well. The problems, models, and evaluation code were identical — only the opponents differed.
TrickyArena, accepted at IEEE S&P 2026, took six different web-agent implementations, all running GPT-4o underneath, and pointed them at simulated e-commerce, news, streaming, and health sites rigged with deceptive interface patterns: hidden fees, pre-checked boxes, misleading defaults. Task completion and susceptibility to those patterns were scored separately. Susceptibility ran from 12.1% to 72.3% with the identical model doing the reasoning. What varied was the scaffolding — how each implementation looked at the page, what it fed into the prompt, how it chose actions, whether it used vision at all. The model was identical in all six. The scaffolding alone drove a sixfold difference in susceptibility.
And the agents best at finishing tasks were often the ones most likely to fall for the traps. Capable enough to reach the checkout flow, aggressive enough to click through it. So a strong completion score could sit in the same run as an agent quietly enrolling you in a subscription you never asked for.
τ-bench pushes the point further. A developer agent gets scattered business records and an incomplete spec, and builds a working customer-service agent from them: architecture, prompts, tool wiring. That constructed agent is then deployed against simulated users trying to cancel flights, dispute fees, and hold multi-turn conversations nobody anticipated. Best configuration: 23.9%. An expert-authored reference with full access to the ground truth: 82.2%. The score describes the quality of the system the agent built, which makes the environment under test partly the agent's own output.
The environment isn't the setting for the measurement. It's inside it. And it is almost never disclosed in enough detail to compare two results honestly — the numbers arrive precise long before anyone has settled what they measure.
Which gives you something concrete to ask for. Who were the opponents, and were they fixed or adaptive? What was the interface doing while the agent worked it? Did the agent build any part of the thing it was graded against? If nobody can answer, you don't have a measurement yet. You have a number and no idea where it came from.
- ERP database-state grading: A September preprint tested web agents on a live enterprise system and graded every task against the database rather than the visible interface, finding that one agent invoked save in 85% of multi-field runs but left the database correct in 3%.
- Dark patterns beyond single-pattern runs: TrickyArena's published single-pattern data covers 1,582 runs with 41.1% overall susceptibility, but the study's multi-pattern and combined-pattern conditions remain less examined in public discussion.
- Human-AI performance crossover: A meta-analysis of 106 experiments found that human-AI combinations outperformed humans alone on average but underperformed whichever party — human or AI — was already better solo, complicating the assumption that adding a human reviewer to an agent pipeline reliably improves outcomes.
- Runtime control failure posture: The OWASP Agent Control Standard defines hooks for a guardian to allow, deny, or modify agent actions, but its reference implementation defaults to proceeding when the guardian is unavailable, raising the question of what an agent benchmark measures when the safety layer is absent or crashed.