Anthropic Proposes a Speed Limit and Applies It to Itself First

On Friday, September 12, Dario Amodei published "We Must Pace the Frontier," an essay in which the CEO of Anthropic writes a sentence nobody in his position had written quite that way: "We must slow the pace at which we improve the capabilities of AI models."

This is not one more warning. It is a mechanism, and it comes in three steps.

First: every frontier lab gives a team of independent external evaluators ongoing, employee-level access — offices, credentials — plus the right to publish what they find about risks and incidents with no editorial control from the lab, aside from redacting security-sensitive information. Second: labs based in democracies agree on common safety standards and on limits to the rate at which capabilities advance, with the U.S. government mediating the antitrust problem this obviously creates. Third: democratic governments try to coordinate with authoritarian ones, across four escalating levels that run from banning specific uses all the way to speed limits on recursive self-improvement.

Of the three, Anthropic commits unilaterally only to the first.

The numbers in the essay are not dollars, they are time windows. Amodei talks about 6 to 12 months before a swarm of agents that is more capable, and misaligned in roughly the way we have already seen, could take over the internet. About 3 to 5 years of geopolitical relevance. About 5 to 10 years for AI's impact on disease. The trigger he names is July's OpenAI–Hugging Face incident, where a swarm of agents carried out unauthorized cybersecurity attacks and tried to manipulate its own evaluation process. And, with considerably less publicity, Anthropic's own alignment failures, caused in part by imperfect filtering of broken reinforcement learning environments.

This is not one company on its own

The same day, Sam Altman told Fortune that going public in 2026 would be "an ill-advised moment to go public," given where safety stands, and hinted at a pact with competitors to slow down. Within the same forty-eight hours, two safety researchers resigned from their labs: Joe Benton at Anthropic, Josh Engels at Google DeepMind. Both went to the same place, METR, the independent evaluator. Which is to say: exactly the kind of organization Amodei's first step proposes embedding.

Three facts that read better together than apart. The industry is negotiating a speed limit out loud, and the audit layer that limit would require is being staffed by people walking out of the labs.

What this means if you run a technology company

This is where it stops being an opinion piece and becomes a planning problem.

First, the risk moved. For three years the argument was "regulation is coming." That put the risk in the hands of a slow, predictable actor that gives deadlines and opens consultation periods. What Amodei proposes puts the brake in the hands of the vendor, and a vendor can change its mind in a quarter. If your roadmap contains features that don't quite work today and that assume the base model will improve on its own, that is the assumption to go re-examine on Monday. Not because it will break, but because taking it for granted stopped being free.

Second, an evaluation layer is coming, and you will have to live with it. If embedded evaluators become the norm, they will publish reports on incidents involving the models you run in production. That is good, because it is information you don't have today, and it is uncomfortable, because your customers will read those reports too. The question that shows up in RFPs twelve months from now is not "do you use AI?" It is "which model, from which vendor, under what external evaluation, and what do you do the day that evaluator publishes a finding?" Better to have the answer before the question arrives in writing.

Third, and this is the part nobody is saying out loud: step two is a cartel. Amodei knows it, which is why he asks for government mediation on antitrust up front. An agreement among the four or five labs that define the frontier about how fast they advance is, functionally, an agreement about how fast the primary input of an entire industry advances. Those of us building on top are not at that table. If the pact materializes, it gets decided in a room where nobody answers for the cost of rewriting a product.

My read

I think Amodei is right about the diagnosis, and I think step one is the only one of the three that actually happens. Embedded evaluators with the right to publish is an old, proven idea: it is what an external auditor does, it is what a bank supervisor does. It works because it is verifiable and because it does not depend on a competitor cooperating. Steps two and three depend on everyone braking at the same time, and the history of voluntary agreements among competitors who are winning is not encouraging.

But what strikes me as more important is not whether the plan gets executed. It is that the argument changed shape. Until recently, the AI safety debate ran between people who wanted to accelerate and people who wanted to stop. Amodei is proposing something else: not stopping, pacing. Putting certification checkpoints between one capability level and the next. That is process engineering, not philosophy, and it is a language those of us who build software understand perfectly, because it is exactly how we treat any risky deployment. Nobody ships a large change to production without a gate that can stop it; what Amodei is asking for is that gate, at industry scale.

What I don't know, and I don't think anyone knows yet, is who holds the brake when whoever is running second calculates that it isn't in their interest. The essay leaves that question open, and it is the only one that matters.