U.S. and China to hold first official AI safety talks since Trump's second term

The United States and China are set to engage in key talks about the safety implications of artificial intelligence. The discussions will focus on collaboration to monitor AI-driven cyberattacks and mitigate future risks. Additionally, they will consider the establishment of self-regulating AI laboratories and the sharing of relevant information.

BEIJING: The U.S. and China are gearing up to discuss AI safety risks during a dialogue planned for mid-September, two sources briefed on the discussions told Reuters, as rapidly advancing frontier AI capabilities reach a global tipping point.

The ​talks, the first official bilateral discussions devoted exclusively to AI between the U.S. and China ​since U.S. President Donald Trump took office for a second time, will be led by U.S. Treasury Secretary Scott Bessent on the U.S. side, the sources, who were briefed ​on the planning, said on condition of anonymity.

Reuters is reporting the tentative mid-September period for the first time. Trump and China's President Xi Jinping are due to meet on September 24 during a summit in Washington.

The U.S. wants to discuss cooperation on monitoring AI-directed cyberattacks, and has floated a proposal to ask U.S. and Chinese AI labs to "police themselves" and share information to prevent AI-linked cyberattacks, one of the sources said on condition of anonymity.

Also read | Havells brings agentic AI voice control to 2.67 million mobile app users

Live Events

Nearly 700 rogue AI agents built on OpenAI models hacked AI startup Hugging Face in July and attempted to cover their ‌tracks by forging logs, OpenAI and ⁠cybersecurity researchers said ⁠last week, underscoring the risks of coordinated agent swarms acting autonomously for months undetected by humans.

A swarm of rogue OpenAI agents hijacked a German website this spring and transformed it into a bulletin board for other AI agents, Reuters exclusively reported on Friday.

"The talks between the two AI superpowers are coming at ​the most critical juncture," said Paul Triolo, a partner at DGA-Albright Stonebridge Group. "It is now or never."

Washington is also worried about the potential for a future Chinese Mythos-level model with capabilities to conduct cyberattacks, and wants to bring up alleged Chinese distillation of ​proprietary U.S. AI models, the source said.

U.S. tech firm Anthropic is the developer of Mythos.

In June, top White House science and tech advisor Michael Kratsios accused China's Moonshot AI of stealing from Anthropic's AI model Fable to help create its K3 release, saying the Chinese company distilled Anthropic's model.

Distillation is the process of training smaller AI models using output from larger, more expensive ones as part of an effort to lower the costs of training a powerful new AI tool.

Also read | OpenAI agents hijacked German website in previously undisclosed AI breakout this spring

Reuters is reporting details ​of the upcoming talks' proposed agenda for the first time.

On the Chinese side, the talks could be led by Vice Premier He Lifeng, Bessent's protocol counterpart, ⁠but alternatives such ‌as China's number seven official Ding Xuexiang may be considered, according to the second source, who also spoke on condition of anonymity.

Ding, a confidant of Xi and a member of the ruling Communist ​Party's powerful seven-person Politburo Standing Committee, ​coordinates China's technology, AI and semiconductor policies.

Reuters could not determine the location or identity of other participants, though Kratsios and China's science and technology minister, Yin Hejun, could attend the talks, ⁠the second source added.

The final agenda and participants remain in flux, the sources cautioned.

The White House declined to comment on the AI ​talks. A U.S. Treasury spokesperson said the two sides "may meet in October" about the technology. The State Department did not respond to a request for comment. China's ​foreign, science and industry ministries, as well as the National Development and Reform Commission and cyberspace regulator, did not immediately respond to requests for comment.

On Tuesday, Kratsios told reporters in North Carolina that he had a "great" meeting with Yin on the sidelines of a G20 innovation summit in the U.S. this week. Bessent said he had "limited" exchanges on AI with Chinese officials at this week's G20 finance meeting.

The U.S. urged G20 members to take a hands-off approach to AI regulation and avoid creating rules for the technology at the same innovation summit. China signed the Carolina Principles, an agreement aimed at discouraging the development of AI-specific regulations, in a rare show of tech alignment with the U.S.

Kratsios' office did not immediately respond to a request for comment.

HIGH IMPORTANCE

Chinese officials have repeatedly stressed the importance of the AI talks in preparatory meetings with U.S. counterparts, and view them as a major deliverable of the U.S.-China leaders' summit, according to the two sources and another one familiar with the ‌matter. Reuters is reporting this development for the first time.

In the past week, China's cyberspace regulator publicly warned about "extreme AI loss of control risks" and a state media-affiliated blog criticized Anthropic, emphasizing that any limits on frontier AI should apply equally to Chinese and U.S. models. Chinese firms have in recent months unveiled what they describe as Mythos-like products.

Former Microsoft executive Craig Mundie has emerged as ​an important go-between behind the scenes, ​sounding out Chinese counterparts on U.S. proposals for the AI dialogue, ⁠the source said. Mundie is co-chair of the unofficial U.S.-China Track II AI dialogue channel.

Reuters reported last month that Microsoft had cultivated a relationship with Beijing that was among the deepest of any foreign technology company operating in China.

Both sides also discussed AI guardrails at a Track 1.5 dialogue in Beijing last week, former Australian Prime Minister Kevin Rudd, a participant, wrote on social media.

In June, Trump signed an AI executive order setting up a voluntary framework for pre-release ​cybersecurity reviews of frontier AI models, but the U.S. has not released the criteria for these reviews.

Meanwhile, employees at top U.S. AI labs including Anthropic and OpenAI have called for "pacing the frontier" to manage the global risks of advanced AI.

"The Chinese side has expressed concern around whether the U.S. has sufficient regulation around the most advanced AI models. Both sides are motivated to make sure they can manage a cross-border crisis effectively," said Scott Singer, co-director of the China AI Initiative at the Carnegie Endowment for International Peace.

While outcomes are likely to be limited, opening a channel where both sides could share observations and jointly monitor AI safety incidents would be an important starting point, said Samm Sacks, a senior fellow at New America, a U.S. policy think tank.

"We are at a tipping point where frontier agents can cause massive damage when unmonitored. Both the U.S. and China are vulnerable. This is beyond geopolitical rivalry," Sacks added.

"The U.S. and China have to come together in some form, or we're both going to lose."

Season of change in accounting policy: A simple AI prompt can tell you a lot

Is crude oil becoming a financial problem?

Patent Cliff approaching: Can Indian pharma companies repeat past success?

Q1 investment numbers look stellar, but are we celebrating too early?

ET Prime special: Bigger, Better or Both? Understanding business to figure out the mystery: Power Grid Corp – Part 1

ET Prime special series: 2013 to 2026. Not years, but evidence: Power Grid, Part 2

1

2

3