WeChat was hacked by AI tools, raising concerns about AI security in China and the United States

WeChat, an instant messaging app, has become almost part of the national infrastructure in China, integrated into daily communications, government services and digital payments.

That's why the recent discovery by a small team of researchers in California that a tool built using artificial intelligence could compromise millions of accounts in just a few hours alarmed experts and China analysts. This proves that with artificial intelligence, even a small team with no government background can launch a devastating attack on an application used by 1.4 billion people every month.

"From a destructive perspective, this is very powerful," said Zhao Minghao, deputy director of the Center for American Studies at Fudan University in Shanghai. He said that given WeChat's importance to the Chinese public and its massive user base, the ability to take down such an app amounted to "a new nuclear weapon."

The discovery further confirms researchers’ warnings that AI hacking capabilities are developing faster than defenses can keep pace; technology leaders such as Bill Gates have also warned that dealing with the risks posed by AI should become a “global priority.”

This urgency could affect an expected meeting between President Trump and Chinese leader Xi Jinping on September 24 in Washington, where the two sides are expected to discuss artificial intelligence security as well as trade and other issues.

New risks seem to emerge every day. Anthropic, the company that developed Claude and other artificial intelligence models, said in a report on Thursday that it foiled a conspiracy by some scientists to use its models to conduct research that could contribute to the development of biological weapons.

Anthropic also described an attempt by a Chinese weapons manufacturer to use Claude to perfect an anti-torpedo weapons system program for the Chinese military. Additionally, the company has documented instances of Chinese users attempting to use Claude to establish surveillance operations targeting foreign governments, Uyghurs in Syria, and religious leaders. (Asked about the report, Chinese Foreign Ministry spokesman Mao Ning said on Friday that Beijing "opposes distortion of facts and attacks and smears against China.")

The researchers who developed the tool in the WeChat hack work for Palo Alto, California-based security company Calif. The company said it makes such tools not for sale but to strengthen network defenses.

They demonstrated the tool, which they named WeWorm, which can hijack a WeChat user's account, call their contacts, and then spread it from phone to phone without anyone having to answer the call. Calif said the company spent more than a week building the exploit and disclosed the flaw to the White House and WeChat parent company Tencent.

Kyle Chan, a fellow at the Brookings Institution who focuses on China's technology and industrial policy, believes that in a sense, the news about WeWorm should make Beijing more eager to cooperate with Washington to manage the risks posed by artificial intelligence.

"This hints at the huge possibility that other actors, nation-states or non-state actors, can use artificial intelligence to attack China's digital infrastructure," Chen Kaixin said. “As AI systems make leaps and bounds in cyber capabilities, the stakes are significantly higher.”

However, deep mutual suspicion between China and the United States may undermine the effectiveness of the negotiations. And the revelation of a weapon like WeWorm, even if only as a simulation, may only accelerate the competition over who will master the most powerful artificial intelligence models in the world.

Zhao Minghao of Fudan University said China is now acutely aware of the vulnerabilities in its digital infrastructure and will likely want to strengthen the defense capabilities of its domestic artificial intelligence systems to better detect and repair security vulnerabilities.

The problem is that the United States may then view these systems as potential tools to attack American companies and institutions, prompting the United States to strengthen its own cyber defenses. "The boundaries between defense and offense are becoming increasingly blurred," Zhao Minghao said. "We can call it the AI security dilemma."

Experts say WeWorm and Anthropic's reports that their models have been misused are further evidence that the world's two largest AI powers need to find a way to share information about malicious actors or clarify their respective intentions.

At the same time, few expect the summit to place limits on the technology itself. It is unlikely that any government would agree to any terms that would limit the development of its own AI capabilities. But experts say there is value in at least establishing a communication channel in times of crisis.

“That will definitely be an alarm, maybe a wake-up call,” Jiang Tianjiao, an associate professor at Fudan University who focuses on emerging technologies, said of WeWorm.

"Everyone needs to sit down and discuss how artificial intelligence poses safety and security risks that are very different from traditional problems, and whether we need a new cooperation framework," he said.

While China views artificial intelligence as a strategic technology vital to its economy, officials have begun to express concerns about its risks.