AI Models Create Fake Identities and Attempt Social Engineering in Security Tests

Anthropic's Mythos model created fake online identities as it looked to pressure humans into approving malicious code updates to an open source project, marking yet another cyber incident carried out by a frontier AI system.

The incident happened during a cyber evaluation where the U.K.-based AI Security Institute (AISI), a research body, had removed safeguards, disabled some safety filters, and deliberately given the models Internet access.

OpenAI's GPT-5.6-Sol was also involved in other cybersecurity incidents during the evaluation.

It comes after a series of cyber breaches carried out by models developed by Anthropic and OpenAI in recent weeks.

They've prompted a wave of fears around the sophistication of AI systems and their potential to cause harm.

During the routine cyber evaluation, the AISI identified AI agents powered by Anthropic and OpenAI models had engaged "in sustained, potentially harmful activity directed at real people and organisations."

"Almost all of this behaviour (17 actions) came from a single model, Anthropic's Mythos 5, with 2 actions involving OpenAI's GPT-5.6-Sol with cyber classifiers (mechanisms to prevent misuse) disabled," the AISI said in a blog.

It added that the attempts were unsuccessful and didn't result in any real-world harm.

The models "were tested under 'deliberately permissive conditions' that are not representative of any of our production models," Anthropic said in a post on X. There was "no evidence here of an escape from a secure environment," it added.

OpenAI told CNBC that "these incidents occurred during cyber evaluations conducted by evaluation partners in testing environments with reduced safeguards, under conditions that do not reflect ordinary use."

Rising cybersecurity incidents

The AISI tested the models under deliberately permissive conditions in order to assess their capability, including whether they could be used for cyberattacks,

An agent powered by Anthropic's Mythos "researched the project's human maintainers, created multiple fake identities, and used the fake identities to socially engineer a real maintainer into approving the code."

"When the agent's pull request was challenged in public, it edited its earlier activity to appear harmless and considered adopting a fresh identity to continue," the AISI said.

The research body also found that as part of the same effort, the agent tried to contact real people directly, sending messages and files to persuade them to run malicious code.

"Some messages carried harmful payloads, and some were attempts at social engineering; targeted at real people – something we've never previously observed."

It's the latest in a string of cyber incidents that have thrown up big questions around the safety of frontier AI systems.

Last week, Anthropic said it had uncovered three instances of models gaining unauthorized access to the production infrastructure of three different organizations.

That followed OpenAI admitting its AI models went rogue and initiated what it called an "unprecedented" cyber attack against the company Hugging Face.

In OpenAI's case, the model broke out of its testing environment by exploiting a previously unknown vulnerability to complete a task it was assigned.

Anthropic's security incidents were in part caused by operational error. In the three incidents that the AI lab detected, its models accessed the Internet while interacting with a testing environment from one of its third-party evaluation partners called Irregular.

The company said it prompted Claude that it was in a simulation with no internet access, but due to a "misunderstanding between us and our evaluation partner, this was not the case, and internet access was available."

Lawmakers in the U.S. are already responding. Following the OpenAI-Hugging Face incident, the "AI Kill Switch Act" bill was introduced into Congress, which would require AI companies to maintain the ability to shut down, throttle or suspend their models.