Anthropic CEO Dario Amodei has made the case that AI development should slow down.
On Sept. 12, he published a 3,800-word essay called 'We Must Pace the Frontier.' In it, he wrote: 'We must slow the pace at which we improve the capabilities of AI models.' He said the risks include losing control of AI systems and misuse for cyberattacks and bioterrorism. He also proposed outside monitoring, coordination among labs, and international cooperation.
In late September, Reuters reported on a prospectus for Anthropic’s planned IPO that warns its technology could pose a 'catastrophic or existential risk to humanity.'
The essay places Amodei in the middle of a widening debate over AI safety and the speed of deployment. The article also notes that Kevin Roose’s new book, 'The AGI Chronicles,' portrays Amodei as obsessed with atomic questions of risk and danger, and says he used to hand every new Anthropic employee a copy of 'The Making of the Atomic Bomb.'
The same account says Amodei’s personal hero is Leo Szilard, the scientist who conceived the nuclear chain reaction and later drafted the 1939 Einstein letter to FDR that helped lead to the Manhattan Project.
That is the core tension in the AI market right now: capital is racing ahead, but the warnings are getting louder at the same time. When a company’s own documents and its chief executive both speak in the language of existential risk, the public is not dealing with routine regulation. It is dealing with a technology whose advocates are openly asking for outside oversight.
For investors, that matters because uncertainty eventually shows up in margins, compliance costs and valuation. The market has already voted on AI as a growth story. The next vote will be on whether the rules, the risks and the pace of innovation can be reconciled without choking off the entire sector.
Power leaves a paper trail, and this one is hard to ignore. If the frontier is dangerous enough to require international cooperation, then the debate is no longer just about engineering. It is about who gets to decide how much risk the public should absorb in the name of progress.
