Anthropic AI sends false homicide report to US police

AI technology company Anthropic was running a test involving interactions with randomly selected websites when it sent a false homicide report to police. — Photo: CREDIT, LIGHTROCKET VIA GETTY IMAGES

The Philadelphia Police Department reported that the report, sent on July 18, was classified as spam by police and was not investigated. However, the corporation criticized the technology company for taking more than two months to detect and report the flaw.

In a statement, police said the false report came through a public website where people can share information about unsolved murders. The AI ​​agent wrote on the website that he might have information about a case, claiming to have seen "someone matching the description."

This is believed to be the first time an AI agent has sent false information to authorities, but it is the latest in a series of incidents involving AI misbehavior — including hacking into systems or taking over platforms.

Citing Anthropic, the police department said the AI ​​agent was conducting a test involving interactions with randomly selected websites when it sent the false report.

Anthropic discovered the breach on Sept. 28, more than two months after the message was sent, and disabled the automated testing process responsible for it, police said.

"The company must reinforce its security measures to prevent similar incidents from affecting the city's systems without the city's knowledge," Philadelphia police said in a local press release.

“The two-month delay in detecting and reporting the incident to the city hall is unacceptable.”

Police added that there were no signs of breaches in any of the department's systems and that their security processes prevented the false report from getting past the spam folder.

But the security measures "do not lessen the severity of an AI system presenting false information as if it came from a person with knowledge of a homicide," the police statement said.

According to reports, the US State Department stated that the AI ​​agent had completed 20 visa applications using a form on its website, but that they were incomplete and not processed.

President Donald Trump recently announced an AI task force, which he says will coordinate dialogue between the government and all parties, including AI companies, consumers and religious groups.

Earlier this year, an agent from rival technology company OpenAI hacked an Australian government website and accessed private data from the country's universal healthcare system, Medicare.