Vision
Vision
The Rise of Ghost Institutions
Somewhere right now, a contract worker is choosing between two AI-generated paragraphs. The rubric says "quality." It says "helpfulness." Nothing about the task registers as a cultural act. In some cases, fewer than a hundred people like this shape how a language model behaves. Meanwhile, AI agents facilitated $22 billion in sales last Black Friday alone.
Those two facts occupy different worlds — one is a gig, the other a market force. But the distance between them is shorter than it looks, and what's accumulating in the gap has a consistency that ought to unsettle anyone paying attention. The people producing it don't have a name for it. Neither does anyone else.
The Rise of Ghost Institutions
Somewhere right now, a contract worker is choosing between two AI-generated paragraphs. The rubric says "quality." It says "helpfulness." Nothing about the task registers as a cultural act. In some cases, fewer than a hundred people like this shape how a language model behaves. Meanwhile, AI agents facilitated $22 billion in sales last Black Friday alone.
Those two facts occupy different worlds — one is a gig, the other a market force. But the distance between them is shorter than it looks, and what's accumulating in the gap has a consistency that ought to unsettle anyone paying attention. The people producing it don't have a name for it. Neither does anyone else.
Default Settings
What Counts as Good When Agents Decide
AI shopping agents describe their recommendations as "high-quality" and "unbiased." Neither term is defined. At small scale, that's a design choice. At millions of daily purchase decisions, it starts functioning like a standard. One that nobody published, nobody voted on, and nobody can fully inspect. Products meeting the criteria get recommended. Products that don't quietly stop appearing. The mechanism by which filtering criteria harden into definitions of quality is worth tracing carefully.
What Stops Being Found When Every Agent Agrees
If different AI agents applied genuinely independent judgment, the filtering described in our companion piece would be uncomfortable but survivable. A product missed by one agent might surface through another. Recent research suggests otherwise: major language models converge not just in what they get right but in what they get wrong, correlating more with each other than with ground truth. The long tail of the web survived on human serendipity and dead-end browsing. Agents don't wander.
Chain of Preference
When an agent recommends a hotel or a vendor, that recommendation arrives carrying the accumulated preferences of people who never thought of themselves as tastemakers.
The chain is long and almost entirely invisible. Data engineers decide what counts as quality training material. Gig workers in Nairobi and Manila label outputs under time pressure, encoding snap preference judgments into reward signals. ML engineers collapse those varied preferences into a single optimization score. Product managers set default parameters and tone guidelines. Enterprise admins scope which tools and catalogs the agent can access at runtime.
Every link is a quiet editorial decision dressed up as a technical one. Nobody in the chain thinks they're deciding what "good" looks like. Every single one of them is.
Further Reading
