False Precision

I have been staring at query results for twenty-five years. The thing that has burned me worst, every single time, is not a wrong number. It's a plausible one. A number sitting in a column with the quiet authority of something that's already been checked. You glance at it, you nod, you build the next decision on top of it. Two weeks later you find out the schema was stale, or the join was wrong, or the fiscal year boundary was off by a month, and now you're unwinding everything downstream.

We've got names for two AI failure modes at this point. Hallucination: the model makes something up with total confidence. Refusal: the model declines to help. Both are visible. Hallucination tends to smell wrong. It invents citations, names entities nobody's heard of, produces answers that are broken in ways that feel broken. Refusal announces itself. But there's a third pattern sitting right there in the open, and I want to point at it, because I think it's the one that actually gets people hurt.

False precision: output that is exact before its meaning is settled.

A SQL query runs. Returns 1,847. A benchmark annotation presents a gold-standard answer. A browser automation agent clicks a button and reports success. Each of these artifacts has the shape of a real answer. Specific numbers. Executed code. Completed actions.

Shape is not meaning. EntSQL, an enterprise text-to-SQL benchmark published this summer, tested leading models on real business-intelligence queries. Every model got the database schema PLUS long-form domain documents describing business rules, metric definitions, fiscal-year conventions. The best model hit 15.9% accuracy. But here's the part that should make your stomach drop: every single query that ran returned a number. Every result set had rows and columns. The outputs looked exactly like analytics. They just weren't analytics. They were noise shaped like signal.

And the benchmarks we use to measure progress on this stuff? An audit of text-to-SQL annotations found error rates of 52.8% in BIRD Mini-Dev and 62.8% in Spider 2.0-Snow, counting mismatches between questions and SQL logic, schema misunderstandings, and domain-knowledge mistakes. The ground truth itself was falsely precise. We were scoring models against answers that hadn't earned their own specificity. The pattern reproduces itself at every layer of the stack.

Web agents show the same thing from a completely different angle. An automated browser agent clicks a purchase button and writes a log entry: element found, click registered, DOM updated, status 200. That log looks like a receipt. I have spent my career reading logs, and I can tell you: that log tells you nothing about whether the system on the other end recognized the agent as an authorized actor, whether the action carried any institutional legitimacy, whether anyone could reconstruct after the fact why it happened. The click compressed perception, decision, authority, and commitment into a single browser event. The success log recorded exactly one of those layers. The other three were never represented.

Here's why false precision is harder to catch than hallucination, and it's almost embarrassingly simple: humans trust specificity. A vague answer invites scrutiny. A precise one discourages it. When a model returns "approximately several thousand," you probe. When it returns 1,847, you move on. The executed query, the row count, the green checkmark in the action log. These are the visual grammar of correctness, and they work on us whether or not they've been verified.

I don't have a detection framework to offer here. That's a different piece. What I want to do is name the thing, because patterns without names are patterns that stay invisible. And invisibility is how false precision actually works. It arrives looking like work that's already been done, like the output of a system that understood the question. Naming it won't fix it. But it changes what you notice. And what you notice, you can at least stop building on top of.