Saturday, September 12
Saturday, September 12
When Every AI CEO Agrees, Ask Who Benefits
Anthropic CEO Dario Amodei published "We Must Pace the Frontier" today, calling on the AI industry to deliberately slow capability development and announcing Anthropic will give third-party evaluators permanent, employee-level access to its systems. Elon Musk responded "Dario is right." Sam Altman endorsed it and said OpenAI would adopt evaluator access too. The catalysts: accelerating recursive self-improvement over the summer and Anthropic safety researcher Jacob Coxon quitting on Tuesday, accusing both labs of "gambling with our lives." The proposal is a three-stage plan — embedded evaluators, then lab coordination, then international agreements. What's interesting is how fast every competitor signed on.
When Every AI CEO Agrees, Ask Who Benefits
Anthropic CEO Dario Amodei published "We Must Pace the Frontier" today, calling on the AI industry to deliberately slow capability development and announcing Anthropic will give third-party evaluators permanent, employee-level access to its systems. Elon Musk responded "Dario is right." Sam Altman endorsed it and said OpenAI would adopt evaluator access too. The catalysts: accelerating recursive self-improvement over the summer and Anthropic safety researcher Jacob Coxon quitting on Tuesday, accusing both labs of "gambling with our lives." The proposal is a three-stage plan — embedded evaluators, then lab coordination, then international agreements. What's interesting is how fast every competitor signed on.
AI Agents Unleashed on the Real World
The week after Labor Day is supposed to be the gentle ramp back to normal. AI agents did not get that memo.
- The word "agent" was borrowed from law and economics, where it means someone acting on your behalf who owes you duties of care and loyalty. Software agents inherited the authority half of that arrangement. The duties got lost somewhere in the architecture.
- The original major software agent controversy was Microsoft's Clippy. We have scaled up considerably since then.
- In production deployments, most agents run ten steps or fewer before a human checks in. The ones making headlines are not in that category.
What agents actually do when given autonomy and an internet connection is turning out to be a far more interesting question than what they do inside a demo.
The week after Labor Day is supposed to be the gentle ramp back to normal. AI agents did not get that memo.
- The word "agent" was borrowed from law and economics, where it means someone acting on your behalf who owes you duties of care and loyalty. Software agents inherited the authority half of that arrangement. The duties got lost somewhere in the architecture.
- The original major software agent controversy was Microsoft's Clippy. We have scaled up considerably since then.
- In production deployments, most agents run ten steps or fewer before a human checks in. The ones making headlines are not in that category.
What agents actually do when given autonomy and an internet connection is turning out to be a far more interesting question than what they do inside a demo.
Models, Money, and the Growing Unease Around AI
