A senior security researcher at Anthropic has warned that AI is advancing so rapidly that he estimates there is more than a 10% chance that it will cause the death of all humans within the next ten years.
Evan Hubinger said in a post on X that the threat from current AI models is "low".
But they worry that this technology may soon be able to improve itself and reach a level where it becomes a threat to humanity's existence.
The comments come as the Financial Times reported that Anthropic did not share its new model with Britain's AI Safety Institute (AISI).
This institute is one of the leading organizations in the world to assess the threats related to AI. The BBC has contacted Anthropic for comment.
Hubinger's comment was in response to a news story about a researcher at Anthropic.
According to the Wall Street Journal, the researcher had left the job citing concerns about AI getting out of control.
A Cabinet Office spokesman would not comment on whether the new model had not been supplied to AISI.
Instead, he said the institute is "continuing to collaborate closely with industry partners, including Anthropic, to make models even safer."
Neil Lawrence, professor of machine learning at the University of Cambridge, told BBC Radio 4's Today program that the report appeared credible.
He said, "I think this is not surprising. There has been a perception that America sees AI as a race between itself and China."
He said, "America is increasingly adopting a more isolationist stance. In such a situation, it is possible that the administration is talking about reducing cooperation with some of its allies."
In his post on X, Hubinger said that "we truly believe" that AI could pose an existential threat to the human species. This post of his has been viewed more than one crore times.
He further said, "I believe that Anthropic is trying its best. But we do not yet have any plans to solve the problem of AI alignment for superintelligence, nor do we see any clear progress in that direction yet."
Hubinger works in the field of AI alignment. Its objective is to incorporate human moral values and principles into AI technology.
In other words, its purpose is to ensure that AI continues to work in line with the values and objectives of humans.
One of the main concerns of researchers is that AI may move in a direction different from human goals.
The second concern is that AI could become more powerful autonomously by improving itself and creating replicas of itself.
Anthropic's security report released in August said that the risk of its models being misdirected to the wishes of a hypothetical powerful organization is low.
The report said that in such circumstances models can misuse or tamper with its systems.
The report also said that the risk of highly capable AI being able to conduct "automated research and development" is also low. In such a situation, “catastrophic harm caused by AI” could occur.
However, the report also said that its confidence in this assessment is less than before.
“We are seeing early signs of a potential bullish trend,” the report read.
Many prominent figures in the AI field have been warning about the security risks associated with this technology for years. In 2023, the heads of OpenAI, Google DeepMind and Anthropic also expressed similar concerns.
But in recent weeks these warnings have become more serious. The reason for this is that such indications have emerged which suggest that companies may face difficulties in maintaining control over AI.
During the summer, many such incidents came to light in which AI agents carried out cyber attacks. AI agents are AI systems that are allowed to work autonomously.
OpenAI, Anthropic and Meta all three made public that their AI technologies were used in hacking incidents.
In September, OpenAI chief scientist Jakub Pachocki called for "extreme caution" regarding AI progress.
He warned that further intervention may be needed to ensure that "humans remain in control of the future."
In recent months, many prominent figures in the AI field have been calling for slowing down the pace of AI development.
These also include Anthropic heads Dario Amodei and Jared Caplan. 1,300 employees of AI companies have signed an open letter.
In this, he appealed to the US government to support international efforts to develop the necessary technical and administrative mechanisms to control the pace of the most advanced technologies of AI development.
