When Human Oversight Works Against You

Every approval gate in an agent workflow is a bet on one person's attention at one moment: that a reviewer meeting this output, somewhere in a day full of similar outputs, will bring judgment worth the interruption. Adding the gate feels like adding safety, so the bet rarely gets examined. It pays off less often than the design assumes, because production review has exactly the features that wear judgment down: volume, repetition, and prompts asking for a decision the reviewer has already made a hundred times.

A meta-analysis in Nature Human Behaviour pooled 106 experiments pairing people with AI systems. On decision tasks, the combined teams did worse than whichever party performed better alone. The gap widened when the AI was the stronger performer: putting a reviewer in front of a well-performing system pulled the result down. In more than 95% of those setups, the human held the final call. On average, the human veto subtracted value.

Why, the study leaves open. Fatigue and review volume weren't among the factors it tested. Research on repeated alerts in other high-stakes work supplies a candidate mechanism.

In a three-week experiment with security warnings, participants were trained to spot app permission requests that should be refused, and started out refusing them correctly 87% of the time. Three weeks of warnings at realistic frequency brought correct refusal down to 55% where the warning always looked the same, and to 76% where researchers deliberately varied its appearance. Trained attention eroded under repetition either way. A study of 1.2 million clinical advisories, the pop-up warnings that fire inside electronic health records, covering 112 primary-care clinicians, found the same shape: each additional alert within a single patient encounter was associated with a 30% drop in the likelihood that the clinician accepted any alert, including the ones that mattered. In neither setting did reviewers disengage. They kept answering the prompts and stopped distinguishing among them.

The evidence has a real seam in it. A 2026 review of 22 systematic reviews on clinical alert fatigue found that only 13.6% of the underlying studies separated appropriate overrides from inappropriate ones. A clinician who dismisses most alerts might be rubber-stamping, or might be filtering noise correctly, and aggregate override rates can't tell you which. The security experiment closes that gap, because those participants were tested on the exact permission risks they had been trained to catch. Their decline wasn't expert triage. It was habituation.

Nobody has published a measurement of this inside production agent approval queues. The structural resemblance is hard to dismiss. Agent review carries the same features: high volume, repeated confirmation prompts, and outputs that are correct often enough to train the reviewer's thumb toward approve.

Review can be designed to resist that: sequencing, triggering on consequence rather than on every action, seeding known-bad cases to find out whether anyone is still looking. But better design still rests on an assumption most teams have never inspected — that reviewer attention scales with the number of things sent for review. It doesn't. Attention spent on confirmations that didn't need it is attention unavailable when something ambiguous shows up.

This is the first full week back after Labor Day, and some organizations are clearing agent output at a pace the alert-fatigue literature would put well past the point of usefulness. The impulse to add one more checkpoint is understandable. It's cheap, it's visible, and it produces a record. A queue where nearly everything gets approved tells you the review step ran. It doesn't tell you anyone looked.