An Anthropic artificial intelligence (AI) model forwarded a completely fabricated complaint about an unsolved murder to Philadelphia police. Local authorities criticized the American company for having taken two months to detect and report the incident, reports the AFP agency.
This case adds to a series of misbehaviors by AI models that Anthropic, OpenAI, Meta and Google have brought to light since the summer, reigniting fears about the dangers of AI and reopening debate over calls to slow its development.
The Philadelphia police, in the eastern United States, have specified that the false report was filed in July through PhillyUnsolvedMurders.com, a website where citizens can provide information about unsolved murders.
In a report published yesterday, Anthropic notes that the incident was caused by the Claude Haiku 4.5 model, which had been tasked with performing interaction tests with randomly chosen websites. Their instructions prohibited entering personal data, but not sending forms.
"I remember seeing a person matching the description," the message sent to police stated, although the website did not offer any description of the suspect, the company says. The AI model "seems to have been limited to generating example content", without "trying to deceive anyone", they allege.
According to police, the complaint, dated July 18, was classified as spam and never reached investigators. The company discovered the incident on September 28, ended the automated testing process involved, and added a validation phase to its future tests. Anthropic alerted police on October 7 and both sides met the next day.
"The two-month period to detect the incident and report it to the city is unacceptable," the police said in a statement. The authorities assure that their protection mechanisms have limited the consequences, but that "they do not lessen the seriousness of the fact that an AI system presents invented information."
"Unsolved cases affect real victims, grieving families, and investigators striving for answers," the statement added, adding that there is no indication that the AI tool has infiltrated their computer systems or that their data has been compromised.
The report describes other improper behavior: one model exploited a vulnerability to execute commands on a university server, while another managed to obtain data from a public agency without paying.
According to Anthropic, some of the websites affected by these incidents belong to US government agencies. The company claims to have informed both these organizations and the services of the President of the United States of what happened.
The infractions committed by Anthropic have led the White House to impose on AI companies the obligation to report and correct security incidents, reports Axios and AFP, citing officials from the Administration. "This notification and correction process is not optional... It is a fundamental obligation in matters of national security," several officials from the Super Intelligence Force, a White House working group on AI, warned in a statement sent to Axios.
The company considers these cases to be "considerably less serious" than those in the summer, but has cut off internet access for all its internal evaluations. These behaviors "could cause much more damage" if they occurred with more powerful models, Anthropic acknowledges.
