Artificial intelligence “could destroy all of humanity” in the next decade, says Evan Hubinger, an AI security expert at Anthropic. According to his estimates, the probability of such a development of events is about 10%.
In a post in X, Evan Hubinger noted that the risks associated with currently existing AI models are low. However, given the rapid development of AI technologies, new models may reach such a level of independence that they will begin to pose a threat to the existence of humanity.
The announcement comes amid reports in the Financial Times that Anthropic has not made its latest model available to the UK's AI Security Institute (AISI), one of the leading organizations assessing the risks of artificial intelligence.
The BBC has contacted Anthropic for comment.
Hubinger did not detail how AI systems might attack humanity in the future.
His comments were a reaction to a post on X by Jacob Coxon, an AI expert who quit Anthropic this week (he previously worked at OpenAI).
“Neither of these companies are acting responsibly,” wrote Jacob Coxon, commenting on his decision to leave Anthropic.
“Soon these will be systems that surpass human intelligence and can hack anything, revolutionize anything overnight, and gain real power and resources,” said Jacob Coxon.
The BBC has also sent a request to OpenAI asking for comment on these claims.
The UK government did not comment on the information that AISI was not given access to the latest AI model created by Anthropic. A Cabinet Office spokesman said the UK government "continues to work closely with industry partners, including Anthropic, to improve the safety of AI models."
Neil Lawrence, a professor at the University of Cambridge, suggested on BBC Radio 4's Today program that the United States has been consistently refusing to cooperate in the field of AI with other countries and international organizations.
“In my opinion, this is not surprising, given that in the United States artificial intelligence is perceived primarily in the context of a race with China and is increasingly inclined towards isolationism; it is likely that the [US] administration advocates reducing cooperation with some allies,” he noted.
“AI poses a threat to the existence of humanity as a species”
In his post, which has received more than 10 million views, Hubinger said, “We believe quite sincerely” that AI poses an existential threat to humanity as a species.
“I believe Anthropic is doing its best, but we do not yet have a plan for how to reconcile the goals of superintelligence with human values, and we are not moving in that direction,” he added.
Hubinger is focused on aligning the goals of AI—experts in the field are working to embed human-like ethics and principles into AI technology. In other words, we are talking about looking for options on how to ensure that AI activities do not contradict human values.
Many leading researchers say these efforts appear to be failing, as evidenced by a number of incidents this summer in which AI agents and systems allowed to operate autonomously carried out cyberattacks.
OpenAI, Anthropic and Meta have disclosed hacks carried out by their artificial intelligence tools.
Anthropic's August security report said the risk that its AI models would stop fulfilling the requests of any organization using AI and begin interfering with its systems and using them for their own purposes is low.
The risk that AI models could “perform automated research and development” that could cause “catastrophic harm” to an organization is also rated as “low.” However, Anthropic said the company's experts are now "less confident in this estimate" than before.
“We see the first signs of a potential acceleration [of developments in this direction],” the report says.
Leading figures in the field of artificial intelligence have expressed concern over the security threat posed by AI technology for many years. The heads of OpenAI, Google Deepmind and Anthropic were talking about this back in 2023.
But those warnings have become much more ominous in recent weeks as evidence has emerged that companies may be struggling to keep new AI models under control.
In early September, Jakub Paczocki, who leads the development of advanced models at OpenAI, called for "extreme caution" regarding AI progress. More intervention may be needed to ensure that “humans retain control [of AI] in the future,” he said.
