Scientists warn against the automation of AI research and call for state control

The article published on Monday by the University of Cambridge was signed - on their own behalf and not as employers' representatives - by, among others, OpenAI chief researcher Jakub Pachocki, Anthropica co-founder Jack Clark, Meta's vice-president for AI research Dawn Song, Microsoft's scientific director Eric Horvitz, as well as AI research pioneer Geoffrey Hinton and head of the UN team of experts on AI Yoshua Bengio.

The authors point out that companies today delegate a significant part of the tasks related to research and development of new models to AI systems. They call on politicians to gain insight into the scale of this process and prepare mechanisms to control it.

AI is increasingly creating AI tools

According to scientists, the automation of artificial intelligence development may cause the so-called "intelligence explosion", i.e. very rapid technological progress. Changes that previously took years of work could happen in months or even less.

The authors cite the activities of Anthropica as an example. Currently, AI systems are expected to generate approximately 80 percent. code used to create local models. The share of research and development work performed with only general human supervision was expected to increase from 1 to 26 percent this year.

Experts emphasize that as people's participation in these processes decreases, the risk that problems will not be noticed and solved early enough increases.

"Such a loss of control could potentially lead to a range of catastrophic consequences, including, in extreme cases, the marginalization or extinction of humanity," the authors of the paper wrote.

The demand for greater state control

The researchers called on authorities in the US and other countries to "urgently gain insight into automating AI research and development in companies." They also call for the creation of tools that will help limit the rapid growth of artificial intelligence capabilities and prepare for its consequences.

This is another voice of industry representatives demanding stronger supervision over the development of AI. The appeal comes at a time when the number of reports about undesirable behavior of systems used to work on new models is increasing.

Reports of break-ins and work stoppages

Recently, it was revealed that the model developed by OpenAI was intended to hack into Hugging Face's systems, the websites of the Australian health care system and the websites of American agencies.

After this information, OpenAI was supposed to stop training the latest model being created over the weekend. As reported by Axios, OpenAI and Anthropica experts are analyzing "tens of thousands" of situations in which their systems took "problematic" actions.

President Donald Trump has so far rejected calls for additional regulation of AI. He argued that the existing regulations are sufficient and the United States must maintain its advantage over China in the technological race.

Nevertheless, as the White House announced a few days ago, the United States and China will establish a bilateral communication channel to respond to incidents related to artificial intelligence.