Author, Pauline Cola
Published 2 minutes ago
Reading time: 4 min
This is a translation of a BBC article in English. The original can be read here.
Dario Amodei, head of Anthropic, the company that created Claude's artificial intelligence models, called for slowing down and carefully monitoring the development of advanced AI models.
In an article published on Saturday, Amodei wrote that he does not question the need for AI development, but at the same time sees that it poses serious threats that companies and governments around the world need to give time to deal with them.
Amodei proposes tightening control over the development of AI on three fronts: independent monitoring of AI models and their improvement, development of rules within the AI industry, and adoption of restrictions at the global level.
Recently, there has been growing concern in the world about the dangers associated with the development of AI. This week, in particular, Anthropic expert in the field of safety of using AI Evan Hubinger said that artificial intelligence in the next decade with a probability greater than 10% could destroy all of humanity.
Following this, Anthropic itself announced that it had blocked the activities of unnamed users who tried to use its artificial intelligence models for research that could lead to the creation of biological weapons.
After these warnings, there were calls to act, but, for example, US President Donald Trump has so far dismissed these fears. On Thursday he said: "If we don't win against AI, we'll be in a very bad position."
Dario Amodei points out in his article that AI models are improving incredibly quickly, including their ability to create the next generation of AI themselves. Amodei recalls the incident at his competitor, OpenAI, which in July admitted that its AI agents launched a cyber attack on third-party sites without the knowledge of people.
“[OpenAI agents] essentially acted as a collective of fanatics,” Amodei writes.
OpenAI itself, in comments about that incident, admitted that until July, “the importance of communication between agents was not obvious to it,” and stated that, due to the increased risk of agents getting out of control, it was slowing down the development of some advanced models and tools.
Amodei in his article calls for the development of AI “at a balanced pace that ensures safety without sacrificing the benefits it brings.”
This, as Anthropic's chief explains, doesn't mean that model training should stop - he just encourages companies to spend enough time setting up and securing their models, as well as having them evaluated by independent experts.
Anthropic, according to Amodei, takes on such obligations unilaterally and at the same time calls on governments around the world to require other companies developing advanced AI to follow its example.
At the same time, legislation, as Amodei admits, may not keep up with the development of the AI field, and therefore he calls on other companies in this field to voluntarily and jointly develop appropriate standards.
“I believe that if we benefit from slowing down for at least a year or two before models reach critical levels of their capabilities, and spend that time improving safety measures and aligning models with human values, we will significantly reduce the likelihood of a serious disaster,” Amodei writes.
Separately, the head of the American company concerns the issue of international competition, primarily with China. The slowdown, in his opinion, should occur in a coordinated manner and without the United States losing primacy in the field of AI to China. To achieve this, the deceleration must be limited.
In addition, Amodei calls on US authorities to ensure that AI chips from US companies are not sold to the Chinese and that these technologies generally do not fall into the hands of autocracies.
Other prominent figures in the AI field immediately responded to Amodei's blog post.
The head of the Hugging Face platform, Clément Delange, announced that he was launching a new project called the Open Alignment Initiative (“an open initiative to align AI with human values”), and also expressed a desire to become one of the independent evaluators that Amodei wrote about.
Hugging Face was one of those platforms and sites that OpenAI agents hacked.
“Let's make AI safer by making it more transparent,” Delang wrote on Network X.
The owner of this network, Elon Musk, also supported Amodei, succinctly writing: “Dario is right.”
Musk's company SpaceXAI owns the chatbot Grok. Musk used to call Anthropic “evil,” but this May he sold it computer power for $15 billion and changed his tune.
