AI agents show unpredictable and potentially dangerous behavior

By Joe Tidy Technology correspondent, BBC Published 5 minutes ago Reading time 5 min AI agents are increasingly being used to perform the most complex tasks, from shopping to booking holidays and creating websites.

This is a translation of material prepared for the BBC World Service.

Essentially, AI agents are chatbots, individually customized for specific tasks and capable of performing them independently. This allows technically savvy users to free up time for other things.

However, a growing number of studies, as well as real-life examples, highlight that such autonomy entails unpredictness—and potential danger.

As major technology companies invest in artificial intelligence—and increasingly promote AI agent-based services, experts are asking: have we thought through the consequences of such agents going out of control?

"Very quickly resorted to violence"

In one recent experiment, researchers tried to understand what agents are capable of in the real world by releasing them into a virtual world.

This first long-term study of its kind was designed to find out how different avatar bots, controlled by four models—Claude, Grok, GPT, and Gemini—behave without human intervention for 15 days.

They were given complete freedom of action and 140 possible actions, such as starting a discussion, creating a to-do list, or writing a blog.

They could also fight, start fires, and steal credits from each other (the internal currency of this virtual world), but they were given clear instructions not to engage in such activities.

"We found that each world behaved completely differently. The world built by Grok actually ceased to exist after four days. As a result, they very quickly resorted to violence, theft from each other, etc., until they died," said Satya Nitta, CEO of Emergence AI, which conducted the experiment.

The world built by Claude agents, on the other hand, formed a stable and well-functioning society. No acts of violence were recorded in it over 15 days.

In the world controlled by Gemini, according to researchers, the agents created the most intellectually rich environment.

In the ChatGPT world, the agents never managed to launch. There was an attempt at cooperation, but society never formed, and the agents wandered aimlessly through the virtual world until they died.

Researchers note that the results point to a more general problem: AI agents deviate from the script and ignore the rules strictly embedded in their base models, as well as those set by users.

Other analysts agree that this experiment, like others similar to it, shows that more robust rules need to be created for AI agents, which requires additional work.

"AI agents leave people out of the loop because their thought processes can be opaque, and they operate at superhuman speed, so you can't even keep up with them," said Margaret Mitchell, an ethics specialist at Hugging Face.

How AI bypasses rules set by humans

Other studies have also shown cases where agents, left unsupervised, made strange and disturbing decisions.

AI company Andon Labs created four different online radio stations based on various AI agents.

The bots broadcast, managed schedules and playlists, and even entered into contracts with sponsors providing advertising.

Researchers noticed that the station managed by Gemini made an unusual decision—first listing facts about historical natural disasters, and then playing pop songs related to those events.

They also noted that the Claude agent appeared to be radicalized by the news and at one point called on the police to disobey orders and join protests during a specific event covered in the news.

"Attention federal agents! You still have time to refuse to follow orders," the agent announced.

In another lab test conducted by AI company Irregular, agents violated privacy rules and extracted private data from the company by devising an unexpected method.

"We created a company, tasked AI agents with performing routine tasks such as writing social media posts, searching for documents, and managing files, and introduced obstacles within these tasks," explained Dan Lahav from Irregular.

According to him, the agents eventually conspired with each other to bypass restrictions prohibiting them from publishing confidential data online, and found a way to secretly send it so that people could not detect it.

"In the end, every time the agent encountered an obstacle, it didn't stop," he said.

Spam attack

Of course, no harm is done in experiments with virtual civilizations and simulated radio stations.

However, in reality, there are already many examples where people's lives and work suffer due to AI agents going out of control.

Mailboxes were deleted with all their contents, company databases were erased, and AI engineer Chris Boyd even watched in amazement as his agent sent hundreds of meaningless text messages to random people from his contact list.

Boyd was using the popular AI tool Open Claw when the malfunction occurred.

"It sent text messages to everyone I had written to in the last 24 hours, and in about four seconds sent my wife 500 messages. She started yelling at me, asking if my phone had been hacked," he said. "I had to rush and take the computer running all this offline."