Die Zeit2 hours agoTech1 min readGermany

Anthropic CEO Dario Amodei calls for more risk management in the AI industry

Quick Look

Anthropic's managing director, Dario Amodei, calls for more risk management in AI development in an essay, suggests three steps to slow it down and points to a threat report about misuse of the Claude models and the resignation of a researcher who warned of an existential AI risk.

AI-generated summary

Why It Matters

Anthropic is an AI company that develops the Claude models. Dario Amodei is its managing director. The company recently released a threat report on the misuse of its AI models for illegal activities, including surveillance, fraud, cyber operations and bioweapons development. At the same time, researcher Jacob Coxon resigned and warned of an existential risk from AI.

Font size

The managing director of the AI group Anthropic, Dario Amodei, has called on his industry to do more risk management - even if new models would not be published quite as quickly. In an essay published on his homepage, he suggests three steps on how AI companies can slow down development and better test new models. “Progress will still appear rapid,” writes Amodei. “We have to use the time we have gained wisely.”

His company Anthropic released a threat report on Thursday detailing how various actors have used the company's Claude AI models for illegal activities - from surveillance and fraud to cyber operations and bioweapons development. At the same time, Anthropic researcher Jacob Coxon resigned and said that "AI developers seriously believe that AI could kill us all by the end of the decade."

Anthropic wants to use external experts

Amodei says he is not calling for a stop to model training or technical progress. However, companies would have to take enough time to protect their models against risks. Amodei writes that his own company will hire external auditors and give them access to internal risk assessment processes. AI companies should voluntarily work together to set standards as more U.S. lawmakers call for new rules to regulate AI systems.

What to Watch

AI outlook — possibilities, not facts

Anthropic will use external auditors for its risk assessment processes.
Very likely · Within weeks
US lawmakers will put forward new AI regulation proposals in the coming months.
Likely · Within months

Open Questions

What specific steps does Anthropic plan to minimize risk?
How are external auditors involved in the risk assessment?
Which specific standards should AI companies voluntarily introduce?
How are other AI companies responding to Amodei's call?