An expert from Anthropic is resigning because he considers the actions of the leading AI companies in the USA to be irresponsible. One of his previous colleagues confirms the concerns about the dangers of artificial intelligence - and his admission triggers violent reactions.
A researcher at the AI company Anthropic has resigned with a drastic warning: The major developers of artificial intelligence were acting irresponsibly "and playing with our lives." Jacob Coxon emphasized on the online platform
“The people developing AI seriously believe it could kill everyone by the end of the decade,” Coxon warned. The 27-year-old Brit told the Wall Street Journal that, according to the “most aggressive scenarios,” things could get out of control by the end of next year. He sees a central danger as artificial intelligence, which can continue to develop itself - and thus becomes more difficult to control.
Anthropic researcher Evan Hubinger, who is responsible for ensuring that AI stays in line with people's interests, confirmed Coxon's concerns a short time later. He personally thinks that the risk of AI killing humanity is more than ten percent in the next decade, he wrote on X. Many users reacted to Hubinger's post with disbelief. "What benefit is great enough to justify putting humanity at 10 percent risk of extinction?" wrote one.
Insiders see dangers - but want to continue
Previously, the head of research at ChatGPT developer and Anthropic's fiercest rival OpenAI, Jakub Pachocki, also urged "extreme caution." Pachocki wrote in a blog post that he is concerned that no one is prepared for the rapid development of artificial intelligence capabilities. We have reached a moment in history in which the intelligence of machines is beginning to surpass that of humans.
The OpenAI manager described in more detail why modern AI systems are harder to control. This artificial intelligence is allowed to “grow” rather than designed. AI systems are a product of complex processes. "Our large-scale training runs are experiments - and we are sometimes surprised by their results." And the more the capabilities of machines exceed those of humans, the more difficult it will be to understand what they are capable of.
At the same time, Pachocki sees an argument for continuing rapid research: developing artificial intelligence that could defend against the dangers of other AI. They will be needed, among other things, to protect infrastructure and develop completely new defense mechanisms.
Hacking attacks as an alarm signal
A wake-up call for the industry in recent months has been hacking attacks in which so-called AI agents independently penetrated other companies' systems in test runs. Such agents are programs with artificial intelligence that are supposed to independently carry out tasks for users. During a test at OpenAI, an AI model found a way to get from what was actually an isolated test environment to the open Internet - and then hacked into the computer system of the AI platform Hugging Face. It only looked for a solution to the test task and did not cause any damage. However, the incident showed how far AI systems can go unplanned on their own without their developers noticing.
Most recently, it also became known that AI agents from OpenAI had already misused a German-language Wiki page on a large scale to collude with each other in the spring. A current protective mechanism used by AI developers is that models with artificial intelligence have to explain their actions in human language. The AI agents used the communication option in test runs to exchange answers to questions posed via notes on the wiki page.
