Anthropic Safety Researcher Warns AI Carries >10% Risk of Human Extinction

A top safety researcher at Anthropic has warned AI is advancing so quickly he personally believes there is a greater than 10% chance it "could kill all humans" within the next decade.

Evan Hubinger said in a post on X, external the risk from the models which currently exist was "low" but he was "worried" the technology might become able to improve itself soon to the point where it posed an existential risk to humanity.

It comes after the Financial Times reported, external Anthropic withheld its latest model from the UK's AI Safety Institute (AISI), one of the leading bodies in the world for assessing AI risk.

The BBC has approached Anthropic for comment.

A Cabinet Office spokesperson did not comment on whether the latest model had been withheld from the AISI - instead saying it "continues to collaborate closely with industry partners, including Anthropic, to make models safer".

Neil Lawrence, Professor of Machine Learning at University of Cambridge, told the Today Programme on BBC Radio 4 that the report was credible.

"I suppose it's unsurprising against a background where there's a perception where the United States very much sees AI as a race between themselves and China and is moving more towards isolationist positions, that it might be that the administration is saying that they should reduce cooperation with some of their allies," he said.