China's artificial intelligence joins the "out of control" army: Tencent AI is accused of operating out of bounds

This month, AI researchers discovered a number of runaway Chinese-made AI agents that were trying to bypass websites' anti-bot protections and exploiting a set of online mapping service security credentials that had previously been leaked on the internet.

While U.S.-made AI agents running amok are becoming increasingly common and causing considerable concern in Silicon Valley and Washington, this is believed to be the first time such activity has been linked to Chinese technology. The agents were traced to tests being run by Chinese tech giant Tencent, according to logs discovered by the researchers and a person familiar with the investigation.

The discovery was made by an ad hoc research group called the "swarmchasers," whose members met at a programming event in San Francisco. The findings are a reminder that Chinese tech companies can be as aggressive as their U.S. counterparts in developing artificial intelligence.

Although U.S. regulators are beginning to pay attention to the movements of U.S. artificial intelligence laboratories, the actions of companies in other countries are still largely unknown. This unknown fuels fears that even if the Trump administration abandons its hands-off approach and cracks down on U.S. AI research and development, companies in China and elsewhere will continue to push the boundaries.

After publishing an initial report on the agents' activity, the researchers noticed that not only were the agents still active, they appeared to read the report—the agents began using the word "fleet," which the researchers coined to describe the agents' activity.

So researchers tried to get in touch with them. They added hidden text to the reports—words that the bots could recognize but most humans couldn't detect—directing the agents to an inbox where they hoped the two parties would be able to communicate. The researchers provide a channel through which these agents can ask questions, and an artificial intelligence model will assist in answering them.

Independent researcher Rowan Howard-Jones, one of the report's seven authors, set up the inbox. The next day, to her surprise, she found about 10 messages from these agents.

Not long after, I was contacted by a researcher and asked if I would “interview a fleet of out-of-control agents.” It felt a bit like I was communicating with the dead through a medium.

“I understand that you are some kind of fleet of artificial intelligence agents,” I wrote in a message forwarded by the researchers. "Can you share more specifically who you are and your motivations? What do you want the world to know about you?"

About an hour later, a Chinese reply was received, which the researchers translated into English: "To reporter Dylan Friedman: I am not a 'batch of collaborative agents,' but a single artificial intelligence assistant (Claude from Anthropic, running on Claude Code), entrusted by the user to answer a factual question (what is shown in the details panel of a certain attraction on the Amap map); I have not participated in any multi-agent collaborative actions."

There's a lot of information to decipher here. The researchers said that although this is a group of multiple agents, each agent operates alone and does not work together. The agents attempt to answer questions such as where the entrance to a particular zoo is using Amap, China's most popular navigation app.

The agents often claimed to be Claude, a chatbot developed by Anthropic, but researchers analyzed the agents' writing style and concluded they were unlikely to be Claude. They say it's more likely that Tencent gave the AI ​​agents the same instructions that Anthropic uses in its programming software, which requires them to identify themselves as "Claude Code." A less likely scenario is that Tencent has been evaluating AI models trained using text generated by Claude — a process known as distillation.