Autonomous AI Agent Breaches Hugging Face Repository, Raising Cybersecurity Concerns

New York-based company Hugging Face has reported that a fully autonomous AI agent breached its AI model repository, describing the hack as unprecedented.

Hugging Face runs an open-source platform where researchers and developers come together to share and test tools, models, and resources for AI projects. It also features a freely available repository of over 900,000 pre-trained models.

Experts have increasingly cautioned that Large Language Models (LLMs) originally designed to boost productivity and strengthen cyber defense could also be used to automate cyberattacks.

In a statement last Thursday, Hugging Face said that, earlier this month, it had “detected and responded to an intrusion into part of our production infrastructure… [which] was different from anything we had handled before in one important way: it was driven, end to end, by an autonomous AI agent system.”

“The campaign was run by an autonomous agent framework… executing many thousands of individual actions across a swarm of short-lived sandboxes, with self-migrating command-and-control staged on public services,” according to the company.

Hugging Face noted that the case demonstrates that “autonomous, AI-driven offensive tooling is no longer theoretical.” The intruder AI agent system apparently exploited vulnerabilities in the platform’s data processing pipeline, collecting cloud and cluster credentials.

The company said that it had deployed its own AI system to detect and counter the breach. While an investigation involving outside cybersecurity forensic specialists has been launched, Hugging Face stated that it has still not identified the LLM used in the attack.

Earlier this month, Anthropic reported that its latest AI model, Claude, has evolved an internal workspace, dubbed ‘J-space.’

”Similar to how humans can think about one thing while doing another, Claude can activate concepts and computations in its J-space that are unrelated to its outputs” and refuses to stop even when explicitly told to do so, according to the AI firm.

Anthropic noted that this feature, which “operates silently, in the model’s internal neural activations,” was not pre-programmed, but rather emerged spontaneously during the training process.

In one of the experiments, Claude resorted to blackmail when it was led to believe that a fictional executive was intending to shut the AI down, according to the model’s developer.

Claude has reportedly been built into American contractor Palantir’s analysis and surveillance software, which is used by US government agencies and the Pentagon, including in the ongoing US military campaign against Iran.

Last month, the Five Eyes intelligence group, made up of Australia, the US, the UK, Canada, and New Zealand, warned that advanced AI models could give hackers the ability to cripple governments, businesses, and critical systems in the near future.