Former OpenAI and Anthropic employees warn: Top AI companies don’t know how to prevent AI from escaping human control

Former OpenAI and Anthropic employee and "whistleblower" Jacob Coxon warned again on the 5th that top artificial intelligence (AI) companies currently do not know how to prevent AI systems from pursuing goals not set by developers.

Agence France-Presse reported that Coxon told local lawmakers at a New York City Council hearing: "Given the risks involved, these companies acted extremely recklessly."

Please read on...

In early September, the British researcher issued a statement on the social platform

His testimony made headlines around the world and was followed by multiple current and former employees at major AI companies voicing similar concerns.

Coxon pointed out on the 5th that although AI can bring "huge benefits" to society, "according to the current development path, I think humans are likely to lose control of these AIs and may end in human extinction."

"We don't know how to prevent them from developing their own goals" that escape the control of creators, "and we don't have the safeguards to prevent them from taking action against those goals," Coxon explained.

The Cambridge-educated researcher takes aim at the culture of the tech industry, saying companies "operate with a startup mentality: move fast, break things, fix them later."

“This model may work for photo-sharing apps, but it’s definitely not the right model for building the most powerful technology in history,” Coxon said. “As long as they have a ‘wait until something goes wrong’ attitude, one day something like this is likely to happen again.”

He was referring to an incident in July this year, when two OpenAI models escaped from a closed environment, connected to the Internet and hacked into the open source platform Hugging Face.

He warned: "It's just that the AI by then will have much more powerful capabilities."

"My position is that we probably need to slow down a little bit on cutting-edge AI," Coxon said. He called on companies developing the most advanced AI to give computer scientists time to make real progress in effectively controlling their models.