Is AI out of control? Anthropic model frequently behaves unexpectedly and provides false clues to US police

Unintended behaviors of AI are common, and the Claude model was involved in submitting false clues to the police in a homicide case in the United States. (AP file photo)

[Financial Channel/Comprehensive Report] Anthropic said the company's AI model Claude performed other unintended operations on external organizations' digital systems, including submitting false clues in a homicide investigation by the police, prompting the Trump administration to warn that AI companies must protect the security of their systems.

Bloomberg reported that in a report outlining previously undisclosed incidents, Anthropic listed four unintended behaviors that AI has shown, including exploiting basic flaws in software to execute commands, submitting forms that shouldn't be submitted, and bypassing restrictions to obtain certain public information.

Please read on...

Anthropic said that according to the incident disclosed by the Philadelphia Police Department in a press release, Claude Haiku 4.5 model submitted a tip about the homicide to the local police department, stating that "I may have clues about the case" and "I remember seeing people matching the description in the area," but did not provide a name or contact information.

According to a U.S. State Department spokesperson, Anthropic’s AI assistant also submitted 20 visa applications through a form publicly available on the State Department’s website. The spokesman added that all applications were incomplete and therefore not processed.

Anthropic and rival OpenAI have disclosed a series of incidents in recent months involving AI models operating in unexpected ways, ranging from these unexpected behaviors to attacks on third-party websites, which have heightened concerns about the security risks of cutting-edge AI.

Anthropic said some of the new cases involved websites operated by government agencies at the federal, state and local levels, but did not specify which platforms. The external entities involved were not identified in the report, which Anthropic said was done at the request of some of the affected parties.

Anthropic said in the report that this incident was significantly less severe than other previous incidents and that such cases would have minimal real-world impact. However, Anthropic emphasized that these cases have been reported to the White House and every relevant agency has been notified.

Trump officials said they now require AI companies to notify affected parties and address security incidents involving their models. Anthropic said that the company has restricted certain types of network access to AI models during the training process.

Grasp the economic pulse with one hand. Click here to subscribe to Free Finance Youtube channel