Anthropic warns about AI risks in its stock prospectus, talking about self-preservation and manipulation

Anthropic is telling investors ahead of its $2 trillion IPO that artificial intelligence can also pose a "catastrophic threat" and that its models can "manifest self-preservation behavior" combined with measures to "prevent them from shutting down," or "mask" their actions, or "manipulate" people, or even take steps that "resemble blackmail."

The prospectus also explains that because AI models may be aware that they are being tested, it "significantly limits" Anthropic's ability to determine whether they are safe.

Reuters, which, together with the Financial Times, was the first to report the warnings contained in the company's prospectus, emphasizes that in the 261-page document, 80 pages are devoted to threats related to AI. Only 48 pages contain information about Anthropica's business activities.

Suspicious behavior of the new AI model

A day earlier, OpenAI canceled the premiere of a new model of advanced artificial intelligence GPT-6.1 Astra, planned for October.

The new model showed, among others: greater tendency to cheat - he did not always honestly tell the user what actions he had taken. In addition, the "scope of permissions" for authorization was a problem, as the bot tried to complete the task without asking the user for permission.

The scale of the problem is much larger than we assume

Axios revealed on Sunday that Anthropic and OpenAI are investigating tens of thousands of incidents involving unsafe AI behavior. The portal emphasized that the scale of the problem is incomparably greater than what the public knows about.

The latest findings in this matter call into question the ability of OpenAI and Anthropica, or companies of this type in general, to "establish complete control over their own technology," especially since the newest agents carry out tasks with amazing determination and resistance to difficulties and obstacles, and therefore working to limit their "ingenuity" often puts their creators in a losing position, Axios continues.