An artificial intelligence "kill switch" may be necessary as the technology advances, according to a co-founder of one of the world's largest AI companies.
Jack Clark is one of seven founders of Anthropic, the San Francisco-based AI firm responsible for online chatbot Claude.
The company was worth $US965 billion ($1.35 trillion) as of May 2026.
Speaking to the BBC, Mr Clark said companies might eventually need a way of shutting off AI software completely if it became too dangerous.
"I've worked in AI for 20 years, and every year [I've] been saying, this technology is getting more powerful by the day," he said.
"And the window to act … is so narrow. It is a few years. We are in this window now.
"Most labs have different ways of being able to pull the plug … but this is the kind of thing you want to feed into the larger policy conversation.
"Should you mandate that companies definitely have a kill switch? Is that kill switch verifiable by a third party?"
Anthropic CEO and 'godfather of AI' concerned
Anthropic chief executive Dario Amodei called last week for the pace of AI development to slow down.
Rogue AI agents, he said, could be capable of "taking over the entire internet" within six to 12 months.
He was then backed up by the so-called godfather of AI, cognitive psychologist and computer scientist Geoffrey Hinton.
"Nobody knows how to estimate the probabilities of these things," he said.
He said governments were regulating AI far too slowly.
"Politicians act very slowly, it's going to be difficult to keep up," he said.
"We've only got a few years."
What is an AI 'kill switch'?
When it comes to just how an AI "kill switch" would work, the answers are vague.
Politicians in the US have proposed a Kill Switch Act, which would require companies to have a way to shut down problematic AI tools.
According to their definition, AI firms should be able to stop an AI's output, "terminate user access", and "shut down" technology when they detected an incident.
University of New South Wales AI Institute chief scientist Toby Walsh said the idea of an "AI kill switch" was simplistic and a "complete distraction".
He also said it could leave systems vulnerable to cyber attacks.
"You put a thing that can turn your computer off; other people can turn your computer off, which is incredibly attractive for bad actors," Professor Walsh said.
Loading...
He said instead of a kill switch, AI companies should face independent scrutiny like industries such as airlines, banks and others.
"The airline industry, people's lives are at stake if airlines break. So, we don't let companies build their own aeroplanes without any independent oversight," he said.
"If you were a human and did what the bots did, broke into someone else's computer, stole passwords, you would be prosecuted."
What happened with OpenAI?
The uptick in concern about AI's going rogue follows Anthropic's rival company, OpenAI, creators of ChatGPT, revealing it had suffered an "unprecedented" incident.
According to OpenAI, two of its most advanced models escaped a testing environment.
The AI agents discovered vulnerabilities in AI development platform Hugging Face and used these to obtain login credentials.
Hugging Face said it detected the breach and had launched a joint investigation with OpenAI.
Professor Walsh said the language used after the hack misled people, and that a lack of oversight was responsible for the incident.
"They talk about [the AI agents] escaping. The AI never started running on anyone else's hardware," he said.
"These frontier AI models require really specialist, expensive, large [graphics processing units] to run on.
"It's not going to be easy at all for them to transfer their weights and start running on someone else's hardware without someone noticing."
What is Australia's AI plan?
Australia has paused "mandatory guardrails" on AI, despite originally planning hard rules to govern the technology.
The National AI Plan instead will use "existing, largely technology-neutral legal frameworks" to manage AI in the short term.
That followed a 2025 call by the Productivity Commission to halt guardrails until an audit could be completed.
Industry Minister Tim Ayres said the plan would make sure technology served Australians, "not the other way around".
"This plan is focused on capturing the economic opportunities of AI, sharing the benefits broadly, and keeping Australians safe as technology evolves," Senator Ayres said.
Professor Walsh said a kill switch would be dangerous in the future, leaving critical infrastructure vulnerable.
"We've seen this in the past where Elon Musk has turned off access to Starlink, which was a vital communication device in the battlefield in Ukraine," he said.
"It creates more of a need for sovereign capability … there is absolutely no way that we can have our national security depend upon the goodwill of the US."
