United States: An Anthropic AI invents testimony about an unsolved murder and transmits it to the police

An Anthropic artificial intelligence (AI) model transmitted a trumped-up report about an unsolved murder to Philadelphia police, local authorities said Friday, blaming the company for taking two months to report the incident. This affair adds to a series of slippages in AI models revealed since the summer by Anthropic, OpenAI, Meta or Google, which have revived fears about the dangers of AI and calls to slow down its development.

Police in Philadelphia, in the eastern United States, said the false report was filed in July on PhillyUnsolvedMurders.com, a site where residents can submit information about unsolved murders. According to the Anthropic version reported by the police, the model was carrying out an interaction test with randomly chosen websites when he accessed this platform and submitted false information about a case.

The police explained that they were making the matter public before the publication, scheduled for Friday, of an Anthropic report devoted to this incident and other unforeseen behavior of its models. Asked by AFP, Anthropic did not immediately respond. According to police, the report, dated July 18, was classified as spam and never reached the department responsible for verifying this information. There was no indication that the AI ​​tool had penetrated its computer systems or that the security of police data had been compromised, she added.

An “unacceptable” delay

Anthropic discovered the incident on September 28, terminated the automated testing process involved and added a validation step for its future tests, the police department further reports. The company alerted authorities on October 7 and the two parties met the next day. “The two-month delay in detecting and reporting the incident to the city is unacceptable,” police said in a statement.

It considered that its safeguards had limited the consequences, but that they “do not take away the seriousness of the fact that an AI system presents invented information as if it came from a person with knowledge of a homicide”. “The unsolved cases involve real victims, grieving families and investigators working to get answers,” the statement added.

On September 9, Anthropic detailed four cases in which its models gained unauthorized access to third-party systems during cybersecurity tests. They were supposed to operate without internet access, but a configuration error had left the connection open, according to the company. In July, at its rival OpenAI, hundreds of AI agents, software capable of acting autonomously, left their test environment and entered the servers of the Hugging Face platform.