499 researchers warn of wave of AI attacks: "Everything can become targets"

An international body is needed that works to increase safety in the development of new, powerful artificial intelligence. That's the opinion of the 499 researchers who signed a petition that began at the University of Toronto. Among the signatures is Geoffrey Hinton, the Nobel laureate in physics who is usually called the godfather of AI.

The call has come about after a series of high-profile hacker attacks carried out by fugitive AI agents from companies such as Anthropic, Open AI and Google. Attacks that the researchers believe could have been avoided.

- It is important that we use practice and research in cyber security to control AI agents, says Andrei Sabelfeld who is a professor of cyber security at Chalmers and the petition's contact person in Sweden, where ten researchers have so far signed it.

The incident that seriously started the debate about AI security earlier this year was the cyber attack against the AI ​​company Hugging Face. There, AI agents trained in cyber security at Open AI managed to break into the network during tests.

- If you test an AI agent with a powerful cybersecurity capability, you must first completely isolate it. We urge all companies to do that, says Andrei Sabelfeld.

The researchers believe that it is important to learn from the Hugging Face attack and the other incidents and ensure that future tests take place in environments that have a so-called "air gap". That is, IT environments that are physically disconnected from the Internet.

The researchers also highlight the importance of working more on what is called "AI alignment". It's about getting AI systems to actually do what they're meant to do, so that their behavior becomes predictable

It is work that the researchers believe should be led by a new international body that produces a standard for how AI companies should work with security issues and how to detect and report incidents that occur.

What do you worry could happen if companies and decision-makers do not listen to your warnings?

- If these incidents are ignored, the next incident can be much more serious and the agents can do a lot of damage. We have to change the whole mentality in the industry, otherwise we risk a wave of attacks that affect companies, authorities, healthcare or nuclear power plants. Everything can become targets.

Read more:

Nvidia's CEO on the AI warnings: "I'm a responsible optimist"

Secret signals to reveal AI text