A Developer Set Up an Optimization Loop With Codex. It Found a 232x GPU Speedup.

The model release calendar has compressed to the point where keeping up is its own part-time job. Some useful context for where things stand:

• Token generation speed has increased roughly 20x since GPT-4's launch in 2023. Most of that gain comes from hardware architecture changes — custom silicon, speculative decoding — rather than model improvements alone.
• The open-weights ecosystem now has more models above 20B parameters than the entire field had published transformer papers five years ago. The barrier to running a capable model locally keeps dropping.
• API pricing is following a pattern familiar from cloud computing's early years: aggressive subsidies from well-capitalized players, margin pressure on everyone else, and customers building dependencies on prices that may not survive the subsidy phase.

Whether you care most about raw speed, local inference, or cost per million tokens, the landscape shifted again over the past 48 hours.

The model release calendar has compressed to the point where keeping up is its own part-time job. Some useful context for where things stand:

• Token generation speed has increased roughly 20x since GPT-4's launch in 2023. Most of that gain comes from hardware architecture changes — custom silicon, speculative decoding — rather than model improvements alone.
• The open-weights ecosystem now has more models above 20B parameters than the entire field had published transformer papers five years ago. The barrier to running a capable model locally keeps dropping.
• API pricing is following a pattern familiar from cloud computing's early years: aggressive subsidies from well-capitalized players, margin pressure on everyone else, and customers building dependencies on prices that may not survive the subsidy phase.

Whether you care most about raw speed, local inference, or cost per million tokens, the landscape shifted again over the past 48 hours.