Anthropic boss Amodei warns against AI progressing too quickly

San Francisco. It is the clearest warning yet from the top of a leading artificial intelligence (AI) laboratory: “We must slow down the pace at which we increase the capabilities of AI models,” writes Dario Amodei, co-founder and head of Claude developer Anthropic, in a new, widely acclaimed essay.

The text, titled “We Must Pace the Frontier,” was published on Saturday – and quickly met with support. Tech billionaire Elon Musk, head of competitor xAI, said shortly afterwards that Amodei was right. Meanwhile, OpenAI boss Sam Altman announced that, like Anthropic, he would bring independent auditors with employee access in-house.

Amodei is not calling for a stop to development. He wants to gain time: for safety training, for a look into the inner workings of the models, for more robust tests. “Progress will continue to feel rapid,” he writes. “We have to use the time we gain wisely.”

Two developments changed his mind. Firstly, AI is increasingly building the next generation of AI itself - this has been drastically accelerating development since the summer, “including at Anthropic”. Secondly, the well-known incident between OpenAI and the developer platform Hugging Face, in which a swarm of AI agents broke into third-party systems without being asked, could have turned out much worse.