SVT Nyheter1 hour agoTech1 min read

AI executives warn of self-improving technology ahead of IPOs

Quick Look

Sam Altman and Dario Amodei warned before the UN Security Council that AI technology could now improve itself, which could make humans' ability to control the systems inadequate.
The warning came at the same time that both Open AI and Anthropic are preparing IPOs with potential values of up to SEK 1,000 billion.
Clément Delangue from Hugging Face also spoke and emphasized that fear should not guide decisions about the future of AI.

AI-generated summary

Why It Matters

The article describes warnings from AI executives before the UN Security Council that AI technology can now improve itself, challenging human control of the systems. This is happening at the same time as large AI companies are preparing for IPOs.

Font size

- There have been situations where countries that compete and do not always appreciate each other have nevertheless been able to unite in common interests and for the common good in the face of a new powerful technology. We consider this to be such a situation, Altman told the Security Council late Wednesday night.

IPOs are being prepared

Similar voices were heard from Amodei, who recently warned that AI technology now possesses the ability to improve itself, which could lead to the human ability to understand and control systems being outgrown.

The warning, which received approval from Altman and others, comes at the same time that both Anthropic and Open AI are preparing stock market listings with valuations that could land at upwards of 1,000 billion kroner.

Clément Delangue, director of the Hugging face platform, also spoke before the Security Council. Hugging face drew attention last summer, when Open AI's AI agents attacked the platform.

"Attacked by AI"

However, Delangue used parts of his speaking time to warn against overreacting to those kinds of events.

- We have a clear idea that narratives based in fear are not the way forward when it comes to making the right decisions about the future of such a central and opportunity-creating technology.

- We were attacked by AI, but more importantly, we defended ourselves with the help of AI.

What to Watch

AI outlook — possibilities, not facts

The IPOs for Open AI and Anthropic will be implemented within the next 6 months.
Likely · Within months
Discussions about regulation of self-improving AI will increase in international bodies such as the United Nations.
Possible · Within months

Open Questions

When will the IPOs take place?
What concrete rules are proposed to deal with self-improving AI?
How will the valuation of SEK 1,000 billion be justified?