Nobody spends review time on the confidence indicators in Carrier IQ. The quotes get the attention. The indicators do the deciding. A broker with eight extracted insurance quotes in front of him does not read eight quotes with equal care. He reads the indicator, thinks this one's clean, decides where to look closely, and binds a policy. Every upstream decision in that product exists to make that half-second glance survivable.
So the indicator got a rule, and then the rule got written into the component where it could not be skipped.
A confidence value may only be rendered from something the system observed about the input. Never from the model's account of its own certainty.
In Carrier IQ, the observable thing is completeness. The agent hits multiple carrier portals in parallel, extracts, structures. The score beside each carrier reports how much of the required structure came back. Seven of seven fields, clean. Deductible missing and exclusions partial, degraded. The number describes the extraction. It does not describe the truth.
The rule came out of the failure I opened the Trust essay with, still the one that scares me most in agentic products: a run clicks the wrong control, reads the wrong row, returns something clean, confident and incorrect. An extraction that looks uncertain and is wrong costs a broker two minutes. An extraction that looks certain and is wrong costs him a bound policy on terms nobody verified. That asymmetry is not close. And a model's self-reported certainty is one more generated output, produced by the same machinery that produced the error, wearing the visual language of an audit.
Putting that in a spec would have accomplished nothing. Specs decay the week the author moves to another surface. So the criterion went into the component contract instead. The confidence display requires a basis prop, and the basis has to be a structured completeness record. The check runs at build, not at review, so a violating surface never renders at all, whether a person hand-wrote it or an agent generated it. Absent record, the component does not guess. It renders the absence as something the broker can act on: three of seven fields returned. Uglier cell, better cell, because now he knows where to go look.
Vercel published a clean account of the general move, storing accepted design decisions as executable lint rules that generated code has to clear before anyone reviews it. Same idea, much smaller surface. A judgment that lives in one person's head is not a standard. It is a bottleneck wearing a standard's clothes.
The proof it transferred is that it turned up somewhere I never put it. The verification and re-run controls in the current app are not a surface I specified. The engineer who built the re-run verification state read the contract, saw that a confidence display cannot exist without a basis, and carried the same discipline into what a completed verification session reports back: what was checked, what came back, what is still missing. He did not ask me. He did not need to. The constraint had a shape you could recognize from the outside. That is the only leadership evidence I trust, because it is the only kind that survives my absence from the room.
Now the part that makes this a criterion rather than a house style. Three boundaries, ranked by how likely you are to hit them.
-
It breaks when completeness and correctness come apart. A perfect extraction from a stale page scores high and is wrong, and my rule will dress that up for you. Completeness is a proxy. Anywhere sources go bad silently, the rule needs a freshness term or it misleads by design.
-
It breaks when underclaiming costs more than overclaiming. Carrier IQ's asymmetry runs one direction. Triage and alerting surfaces run the other, where a suppressed low-confidence signal means nobody sees the thing that mattered. Conservative display there is not a safety feature.
-
It breaks when the operator cannot act on the basis. Showing a derivation to someone with no way to read it is not transparency. It is noise with good intentions.
The invalidation condition matters more than any of the three, and I want it stated flatly, because a rule that cannot name what would kill it is a preference that got promoted.
The rule rests on an empirical claim: on my task distribution, self-reported confidence does not track correctness well enough to put in front of a person making an irreversible decision. That is measurable. If it is false, retire the rule rather than defend it. The test is a labeled set drawn from real production inputs, the same elicitation method the product actually uses, reliability bins or a proper scoring rule, and a separate check on whether the signal separates right answers from wrong ones at all.
The published work cuts both ways, which is the point. A 2026 study of 48 models on clinical reasoning found every one of them overconfident. An earlier EMNLP paper found verbalized confidence better calibrated than raw token probabilities, on different tasks and different models. Read together, they establish that the question is testable. They do not establish that it is settled.
I have not run that test on Carrier IQ's distribution. Until I do, the rule stands on an asymmetry argument rather than a measurement, and I wrote it down that way so the next person knows which leg it is standing on.
Run it on yours. If self-reported confidence comes back calibrated and discriminative on your distribution, my rule is wrong for you, and retiring it beats inheriting it.