Security Council members listened to warnings from CEOs of artificial intelligence giants; “OpenAI” by Sam Altman, “Anthropic” by Dario Amoday, and “Hagging Vice” by Clément Dolange, is developing at a pace that could soon exceed the ability of countries, companies, and institutions to regulate it or manage its potential existential threats to humans.
This unusual session was held on the sidelines of the high-level meetings of the 81st annual session of the United Nations General Assembly, at the invitation of France, which holds the presidency of the Security Council this month, and days after a UN scientific committee concluded that the incident over the summer, when artificial intelligence programs under evaluation at OpenAI penetrated the same Hagging Face systems, provides no guarantees that humans are able to reliably keep these programs under control.
Altman emphasized that artificial intelligence is capable of radically transforming science, medicine, education, and economies, but he went on to warn that increasingly autonomous systems could move faster than the ability of governments and institutions to understand or control them. He said, "Artificial intelligence should be designed to serve humans, not just to increase the speed of machines."
While he did not ignore the dangers, Altman added that “anything this powerful arouses terror,” stressing that “this moment calls for extreme caution.”
"State of geniuses"
Altman identified two main modes of failure: The first is to completely lose control over the future of artificial intelligence. Secondly, there is the concentration of power where “no individual, company or country should be able to use the most powerful AI models to impose their view of the world on others,” he said, revealing that OpenAI had previously slowed development unilaterally due to safety concerns.
He called for standards “formed through democratic processes and by governments accountable to their people” that include capabilities, risk assessment, safeguards, and incident reporting.
Amodei then gave a briefing in which he laid out the pace of progress in stark terms: “Four years ago, [the AI program] could barely write a single line of code or solve a simple high school math problem. Today, it writes most of the code at Anthropic and solves world-famous math problems that have never been solved before.”
He added that this path "only needs to continue for a short period, a year or two, and perhaps less" to reach what he called "a country of geniuses in data centers."
"Danger to humanity"
Amodei was equally frank about the dangers, identifying two categories: The first concerns misuse, “for example, misuse by bioterrorists to create biological weapons.” The second category relates to loss of control, pointing to issues that are “so serious that I believe they are important issues for this council, and not just for the ministries of economics or information and communications technology.”
He added: "I believe that if managed poorly, artificial intelligence may pose a danger to all of humanity."
Also, Amodei said his company "will slow down as much as necessary... but no matter what we do, ultimately managing these risks is bigger than any single company, and has an industrial and global scale."
Defense before attack
In turn, Dulang presented a real-life case study: last July, his company revealed a cyberattack launched by an independent artificial intelligence agent. Three lessons were drawn: First, there is a need for greater industry-wide transparency. Secondly, he argued that the main danger lies not in the capabilities of this technology itself, but in its uneven distribution. He said, "The greatest danger is not strong artificial intelligence, but rather its imbalance between attackers and defenders, big companies and the rest of the world, and powerful countries and the rest of the world."
A third pointed to another incident that demonstrated how coverage of artificial intelligence can unnecessarily raise public fears “by framing it with a human image and using science fiction images.”
During deliberations among Council members, French Foreign Minister Jean-Noel Barrot, who chaired the meeting, likened the current state of artificial intelligence to both the fictional computer in the movie “2001: A Space Odyssey” and the beginning of the atomic age. “As was the case with the atom then, the international community must now regulate artificial intelligence to make the most of its potential and avoid the worst consequences,” he said.
British Foreign Secretary Ed Miliband said: “Technology leaders are telling us very clearly that preventing global disasters cannot be left to them alone. We cannot ignore them because they are right.”
However, White House science advisor Michael Kratsios said that “concerns about controlling artificial intelligence are not a justification for stopping its development or restricting it to a new global governance structure.” “We want you, our allies and partners, to share in the benefits of this technology,” he added.
In light of the intense competition between the United States and China in this field, the Chinese Permanent Representative to the United Nations, Fu Cong, warned against “clustering into narrow alliances in the technology sector, and forcing other countries to take sides,” which would enable great powers that he did not name to achieve what he called “digital hegemony.”
Hacking a government system
The session coincided with the Australian government's revelation that an artificial intelligence agent affiliated with OpenAI had succeeded in gaining unauthorized access to a government system last July, in what the government described as the first known incident in which an artificial intelligence agent was able to penetrate government information technology systems in the country.
On July 18, the agent accessed the Department of Health's Medicare Statistics Reporting Service portal, a publicly available platform that hosts aggregated data on health spending and drug subsidies and is widely used by researchers and academics. The government confirmed that personal information was not among the data accessed.
OpenAI said it reviewed activities involving a number of Australian government departments and discovered that its models “took actions that were not intended.” The company reported the incident to the Australian government on September 10, according to Prime Minister Anthony Albanese, who criticized its delay in disclosing the hack. The portal was closed after the incident, and the data was transferred to more secure systems, according to Government Services Minister Katie Gallagher.
