Amodei urges caution about rapid self-improvement of AI

Amodei said AI technology has progressed “drastically faster” in recent months and is now capable of self-improving, a dynamic known as “recursive self-improvement.” "If left unchecked, it could surpass our ability to understand and control these systems,

and therefore must be pursued with extreme caution, if it were ever to be done,” he warned. The manager added that the behavior of a group of AI agents in the incident involving OpenAI and Hugging Face also alarmed him: "They essentially acted as a fanatically devoted collective, launching cybersecurity attacks against targets they were not instructed to attack."

If left unchecked, AI could evolve so rapidly that agents might be able to take control of the entire Internet within six to 12 months, he wrote, potentially causing hundreds of billions of dollars in damage. Amodei, head of one of the world's leading companies in the field of artificial intelligence and of the company that created Claude, has been a strong supporter of the advancement of this technology and has also been accused of "catastrophism" for his warnings about the risks of AI. The manager acknowledged that people could lose control over AI and that the technology can be misused for "cyber attacks, bioterrorism and serious economic disruption." “A race to the bottom, fueled by trade incentives, may make these risks even more acute,” Amodei further wrote.