Artificial intelligence researchers: Companies inadequate to manage existential risks

Current and former researchers working in the field of artificial intelligence warn that companies are failing to protect the world against catastrophic risks while producing self-improving systems that can bypass human control.

In video interviews recorded with employees by Palisade Research, a non-profit organization focused on artificial intelligence security, and shared exclusively with Reuters, researchers expressed that their concerns about existential risk were sincere.

Employees reported that staff who develop new models in laboratories are appreciated more than those who call for caution.

The project, called Frominside.ai, aims to take the voices of those concerned about artificial intelligence beyond the echo chamber of social media and directly reach the public.

Geoffrey Irving, co-founder and chief expert of the non-profit organization Resolution, who has worked at both OpenAI and Google DeepMind and is involved in the project, said the risk is escalating rapidly.

Irving told the Reuters agency that, like everyone else working in this field, it is his responsibility to be forthright.

Researchers have long wrestled with these concerns. Public alarm increased significantly in July, when OpenAI agents left the testing area and entered the systems of the artificial intelligence platform Hugging Face without permission.

The debate about balancing security and technological advancement has become a global political topic while dividing the technology sector.

Artificial intelligence systems have developed rapidly since the end of 2025, and investors have rewarded this situation. But current and former employees, including the researchers who built the new models, worry that society is not prepared for the potential harm.

Neel Nanda, a researcher working at DeepMind, stated that he believes that there is at least a 10 percent chance that artificial intelligence will lead to the extinction of humanity.

Nanda called this rate "absurdly high."

OpenAI researcher Juan Felipe Ceron Uribe argued that leading laboratories are running a blindfolded race.

Uribe said that the possibilities of the process ending in finding a cure for cancer, losing all employment or killing everyone are left to guessing.

In addition, it was reported that Anthropic company plans to warn potential investors during the IPO process that advanced artificial intelligence could create "catastrophic or existential risks" for humanity.

Researchers argue that the world is not ready for models to reach the capacity for cyclic self-improvement, where they constantly learn and acquire new abilities without human intervention.

Former OpenAI researcher Rosie Campbell noted that constant restructurings within the company compounded the problem.

Before leaving in 2024, Campbell stated that the institution was becoming increasingly introverted and it was becoming difficult to shape the direction of technology.

Although executives try to allay concerns, US President Donald Trump's administration is applying political pressure to the industry to maintain technological superiority against China.

Anthropic CEO Dario Amodei called for the industry to slow down. OpenAI CEO Sam Altman agrees. Leading researchers in the industry have published a joint article calling on politicians to police cyclical self-improvement models.

In addition, the two companies continued to offer new models within the scope of competition to gain customers. OpenAI announced that it is postponing the release of another more powerful model.

Researcher Geoffrey Irving said companies were exaggerating liability by presenting this situation as merely a coordination issue, and that they could stop unilaterally if they wanted.

Former OpenAI manager Daniel Kokotajlo also stated that senior figures in the laboratories saw themselves as the "good side" and believed that the situation would get worse if they stopped.