Author, Kali Hays
Role, BBC technology reporter
and
Author, Faisal Islam
Role, BBC Economics Editor
Published 3 minutes ago
Reading time: 4 minutes
The co-founder of Anthropic, one of the artificial intelligence (AI) companies, told the BBC that in the future, companies may need to make it mandatory to equip an artificial intelligence "emergency shutdown" mechanism (kill switch) that can be inspected by a third party.
Jack Clark, one of the seven founders of Anthropic, said that if AI software becomes too dangerous, there must be a way to shut it down completely. Society "might finally need to set rules".
Clark said that "most labs, including Anthropic, have different ways of shutting down systems," but he thinks lawmakers may need to mandate that companies set up such mechanisms in the future.
With the rapid development of AI technology and rising concerns about the risks it may pose to humans, a series of warnings from senior executives and employees of AI companies have pushed related issues to the focus of public opinion.
Some people have publicly stated that if left unchecked, this technology may eventually lead to the demise of all mankind.
Anthropic CEO Dario Amodei reiterated his call over the weekend to slow down the pace of artificial intelligence development and tighten regulation. This is consistent with the company's past stance, although some have questioned the motives behind these calls.
Amodei added that any measures to limit the development of artificial intelligence should be implemented "without sacrificing commercial advantages."
Clark told the BBC that the specific requirements and verification methods surrounding the "emergency shutdown" mechanism should be part of the broader policy discussion on artificial intelligence.
He said: "Should businesses be required to have an emergency stop switch? Can this switch be verified by a third party?"
"I think this type of issue is what society wants to understand and may ultimately want to regulate through legislation."
Anthropic, founded in 2021 by a group of former OpenAI employees, is at the center of the AI safety debate.
Last week, an artificial intelligence researcher who left Anthropic because of concerns that AI may lead to human extinction posted a post that attracted widespread attention.
In response, Anthropic scientist Evan Hubinger said he personally believes the likelihood of AI causing human extinction within the next decade is "more than 10%."
Geoffrey Hinton, a computer scientist and Nobel Prize winner known as the "Godfather of Artificial Intelligence," told the BBC on Friday (September 11) that the belief that AI has a 10% chance of wiping out all humankind is "not unreasonable."
He and others with similar concerns believe that AI could have catastrophic consequences by taking over critical systems connected to the Internet and turning them against humans.
However, some people in the artificial intelligence industry believe that concerns about AI destroying humanity are exaggerated and may even be used to create buzz.
"Sorry, but asking Jacob Coxon (an AI researcher who left Anthropic) about the risk of AI extinction is like asking an air conditioner repairman about climate change," Clement Delangue, head of Hugging Face, a developer platform that was hacked by OpenAI bots, said last week.
"I'm not saying it's necessarily uninteresting or wrong, but we should still look at the issue rationally."
George Arison, CEO of dating app Grindr, said concerns about AI tools are being used by companies to support their business plans.
Anthropic, which is valued at US$965 billion (6.47 trillion yuan; NT$30.53 trillion), is preparing for an initial public offering (IPO) that would allow investors to buy shares in the company.
OpenAI, which was last valued at US$852 billion (5.71 trillion yuan; NT$27.07 trillion), was expected to do the same, but OpenAI CEO Sam Altman said on Friday that it would not move forward with its listing plan this year due to the ongoing debate around AI safety.
"The only way to justify these valuations is to say, 'I'm going to take over all the industries, I'm going to take over all the jobs, and my AI will do it all,'" Arison said.
Asked about the chances of AI wiping out all of humanity, Clark said: "I don't think statistics like that matter much."
However, he added that allowing AI to continue to develop as a "completely unregulated industry" is not a good idea.
"We are rolling a very risky dice," Clark said.
"The point is, we have to change the direction of this industry."
U.S. lawmakers have proposed a bill called the "Kill Switch Act" that would require companies to have mechanisms to shut down problematic artificial intelligence tools.
The bill will also give some government agencies the power to require relevant artificial intelligence tools to be shut down or restrict their operation.
However, US President Trump has rejected calls to slow down the development of artificial intelligence. He said on social media: "Artificial intelligence taking over the world, destroying humanity, and all other terrible things are a hoax (HOAX)."
Later, Trump compared related concerns to the "Global Warming Scam not long ago" and said that people were also told at that time that they would die from extreme heat.
He wrote in an earlier post: "There is a sick conspiracy against artificial intelligence and data centers, and the only one happy about it is China. Whoever wins artificial intelligence wins everything!"
We used artificial intelligence to help translate this article, and its original text is in English. Before publication, BBC reporters checked the translated content.
