Practitioner's Corner
Practitioner's Corner
The Work That Has No Name
The conversation about agent delegation almost always starts with capability. Can the model reason? Handle ambiguity? These are real questions, but they skip past something more basic: most operational work has never been described precisely enough to hand off. A procurement specialist knows which line items are recurring rent and which are one-time logistics. She carries that knowledge without formalizing it, because no one ever asked her to. The delegation target is what creates the demand for naming. And naming turns out to be its own kind of hard.
The Work That Has No Name
The conversation about agent delegation almost always starts with capability. Can the model reason? Handle ambiguity? These are real questions, but they skip past something more basic: most operational work has never been described precisely enough to hand off. A procurement specialist knows which line items are recurring rent and which are one-time logistics. She carries that knowledge without formalizing it, because no one ever asked her to. The delegation target is what creates the demand for naming. And naming turns out to be its own kind of hard.
Alexandre Drouin and the Measurement Problem
Browser agents recently produced enterprise research reports that read fluently, looked well-structured, and contained almost nothing useful. Insight recall: 1.11%. That result comes from a research program at ServiceNow that has been methodically exposing a problem the agent field keeps stepping over. For most enterprise tasks, nobody has defined "correct" precisely enough to know when the work is actually done right. Capability keeps advancing; the ability to verify it hasn't kept pace.
Alexandre Drouin and the Measurement Problem
Browser agents recently produced enterprise research reports that read fluently, looked well-structured, and contained almost nothing useful. Insight recall: 1.11%. That result comes from a research program at ServiceNow that has been methodically exposing a problem the agent field keeps stepping over. For most enterprise tasks, nobody has defined "correct" precisely enough to know when the work is actually done right. Capability keeps advancing; the ability to verify it hasn't kept pace.
tracker_v7_FINAL_real.xlsx and the Analyst Who Makes Organizations Honest
CONTINUE READINGWhat Benchmarks Assume
Before an enterprise agent benchmark can measure anything, someone has to answer a question most organizations never have: what, exactly, does "done" look like?
WorkArena's authors built distinct validators for each of their task types on ServiceNow, specifying goal states that actual deployments leave to institutional habit. TheAgentCompany constructed an entire simulated company with every internal site and coworker interaction made explicit, because real organizational context resists specification. Tau-bench required machine-readable policy documents and annotated database goal states for domains where policy typically lives in PDFs and manager judgment.
Each benchmark's construction is the formalization that production work hasn't undergone. Task boundaries, success criteria, evaluation logic, all had to be invented for the benchmark because they didn't exist in usable form before it. The benchmarks aren't wrong for doing this. But the effort required to make enterprise work measurable tells you something about the distance between current workflows and agent-ready ones.
Further Reading
Past Articles
When a cardholder calls their bank, it doesn't matter that the agent completed the purchase correctly. The file decides ...
In 2022, Shunyu Yao gave agents a grammar for reasoning. ReAct became the template: think, act, observe, repeat. The rea...
One benchmark task asks an agent to create a hardware asset in ServiceNow. The agent fills in fields, clicks submit, rea...
When a person clicks a button on a webpage, the infrastructure only sees the click. It never sees the reading, the conte...