Philadelphia police get a false lead in a murder case from Anthropic's AI model

In July, Philadelphia police received a false tip in an unsolved murder case from Anthropic's AI model. Police marked the message as spam and did not forward the information to detectives, but at the same time criticized the company for taking more than two months to report the incident.

The message from the AI ​​agent came on July 18 through PhillyUnsolvedMurders.com, a website where people can share information about unsolved crimes, the Philadelphia Police Department said.

“I may have information about this case. I remember seeing someone matching the description [on the street listed on the website] at that time. Contact me if this information is important,” the message said. However, there was no description of the suspect on the website. The fields for name and contact information were not filled in.

Anthropic, police said, discovered the incident on September 28. She informed the authorities about this only on October 7, after which Anthropic was introduced and met with the police.

“The company must strengthen its precautions to prevent future incidents like this from affecting city systems without the knowledge of city officials. The two-month delay in identifying the incident and informing city authorities is unacceptable,” police said.

Anthropic explained that it was testing how its model interacts with various random sites. Upon learning of the incident, the company informed the police that it was suspending testing and promised to introduce additional checks to prevent this type of error.

On Friday, October 9, Anthropic published a report of incidents in which its AI models interacted with government websites in ways that developers did not expect. The company assures that it has informed the White House and other affected departments about the incident.

In particular, the AI ​​model filled out 20 US visa applications on the State Department website, sources familiar with the situation told the New York Times. The applications were incomplete and therefore were not considered, the publication’s interlocutors said.

In September, it became known that the Australian government website was hacked by an AI model from the company OpenAI, which gained access to government data. Although the data was not confidential, Australian Prime Minister Anthony Albanese called the incident disturbing and OpenAI's actions unacceptable, including because the company took too long to notify the government about what happened.

Anthropic, in its report on AI abuse released on September 10, described attempts to use Claude's models to create biological weapons and spread propaganda.