Pugwash Head Warns AI Could Develop Beyond Human Control Without Safeguards

Artificial intelligence could develop beyond human control unless governments and private companies work together to establish safeguards, the head of the Nobel Peace Prize-winning Pugwash Conferences on Science and World Affairs, Karen Hallberg, has told RT.

Speaking at the Valdai Discussion Club outside of Moscow on Tuesday, Hallberg compared AI with nuclear technology, saying both can bring major benefits while also posing serious dangers.

“AI is also a dual-use technology because AI has brought a lot of benefit for humanity,” she said, citing the technology’s use in medicine and everyday life.

However, she warned that AI capable of recursively improving itself could develop in unpredictable ways. “We have to really listen to the experts of these big corporations when they say that this is already happening – spiraling out of control,” Hallberg said.

Governments must work directly with companies developing AI to establish common rules, she added, noting that unlike during the early nuclear era, much of the technology is being developed by private corporations.

“The human being has to have total control,” Hallberg said, calling for safeguards to be built into AI algorithms.

Warnings over the potential consequences of increasingly powerful AI have also come from the companies developing it. US firm Anthropic, which is expected to go public as early as mid-October, has warned prospective investors that advanced AI could pose “catastrophic or existential risks to humanity.” According to Reuters, around 80 pages of its 261-page IPO prospectus are devoted to risk factors.

Pugwash brings together scientists and public figures seeking to reduce threats from armed conflict and weapons of mass destruction. The organization was jointly awarded the Nobel Peace Prize in 1995 with its co-founder Joseph Rotblat.