© TinyFish, Inc.. All rights reserved.
Evidence Across the Stack
An agent updates a billing tier, submits a regulatory form, merges a PR. Three days later someone shows up asking who authorized it. You reach for what you have: traces built for engineers debugging execution, outputs built for users seeing results. Neither answers the question. Both look like evidence because they're detailed, but specificity isn't verification. The evidence locker is a third artifact, designed for the stakeholder who arrives after the fact needing to accept, contest, or unwind what an agent did. Its contents scale with how hard the work is to reverse.
An agent updates a billing tier, submits a regulatory form, merges a PR. Three days later someone shows up asking who authorized it. You reach for what you have: traces built for engineers debugging execution, outputs built for users seeing results. Neither answers the question. Both look like evidence because they're detailed, but specificity isn't verification. The evidence locker is a third artifact, designed for the stakeholder who arrives after the fact needing to accept, contest, or unwind what an agent did. Its contents scale with how hard the work is to reverse.
An agent runs a query. $14.2M comes back, clean execution, no errors. Looks like an answer. But the best-evaluated system in the EntSQL benchmark hit 15.9% accuracy when enterprise domain knowledge was required, and the benchmarks themselves carry annotation error rates above 50%. That number isn't an answer. It's a claim wearing a suit. What follows builds the evidence bundle that makes such a claim contestable, element by element, each layer entering because the previous one left something dangerously exposed.
An agent runs a query. $14.2M comes back, clean execution, no errors. Looks like an answer. But the best-evaluated system in the EntSQL benchmark hit 15.9% accuracy when enterprise domain knowledge was required, and the benchmarks themselves carry annotation error rates above 50%. That number isn't an answer. It's a claim wearing a suit. What follows builds the evidence bundle that makes such a claim contestable, element by element, each layer entering because the previous one left something dangerously exposed.
An agent run produces a trace. It logged the calls, returned the output, maybe even succeeded. But a trace tells you what happened. It doesn't tell you whether the outcome was allowed to stand.
That second question demands a record, not a log. And the shape of that record depends entirely on what kind of agent ran and what kind of scrutiny might follow. A dashboard annotation and a wire transfer don't warrant the same evidence bundle. Below are five cards, one per agent type, listing the minimum artifacts that should survive a run. Think of them as checklists shaped by the specific challenge each agent type will most likely face later.
An agent run produces a trace. It logged the calls, returned the output, maybe even succeeded. But a trace tells you what happened. It doesn't tell you whether the outcome was allowed to stand.
That second question demands a record, not a log. And the shape of that record depends entirely on what kind of agent ran and what kind of scrutiny might follow. A dashboard annotation and a wire transfer don't warrant the same evidence bundle. Below are five cards, one per agent type, listing the minimum artifacts that should survive a run. Think of them as checklists shaped by the specific challenge each agent type will most likely face later.
Past Articles
Agent outputs keep getting more detailed, more structured, more confident-looking. That's exactly the problem. A text-to...
Hallucination has a name. Refusal has a name. There's a third AI failure mode nobody's bothered to name yet: output that...
Anyone can physically sign a document. The motion is trivial. What makes a signature consequential is everything the mot...
An MCP tool call carries a name, some arguments, and gets back a result. That's the complete method-specific payload. Th...
