Breaking
BRDrone shows damage to factory after fire and explosion in Salto; see before and afterTRIt starts tomorrow! Penalties are on the way for drivers who do not install winter tires...BRFirefighters search for driver of car that fell from bridge into river in Cruz das PossesCNThe 26-year record ends! US loses measles-free statusDEDeep red into the Nasdaq debut: Will Anthropic's risky stock market bet pay off?USWhere to watch UFC 334: Fight card, start time and more key info for tonight on Paramount+BRKnown for selling popcorn in the neighborhood, woman disappears after leaving gym in Campo GrandeUSOhio State vs. Northwestern odds, prediction, time: Week 11 college football picks from proven modelDEPlans for deductions for early retirementBRTrio escapes from prison in Curado; video shows man being captured near the unitBRDrone shows damage to factory after fire and explosion in Salto; see before and afterTRIt starts tomorrow! Penalties are on the way for drivers who do not install winter tires...BRFirefighters search for driver of car that fell from bridge into river in Cruz das PossesCNThe 26-year record ends! US loses measles-free statusDEDeep red into the Nasdaq debut: Will Anthropic's risky stock market bet pay off?USWhere to watch UFC 334: Fight card, start time and more key info for tonight on Paramount+BRKnown for selling popcorn in the neighborhood, woman disappears after leaving gym in Campo GrandeUSOhio State vs. Northwestern odds, prediction, time: Week 11 college football picks from proven modelDEPlans for deductions for early retirementBRTrio escapes from prison in Curado; video shows man being captured near the unit
BackMicrosoft found a way to turn a helpful AI into a sleeper agent
Microsoft found a way to turn a helpful AI into a sleeper agent
Tech
CryptoSlate2 hours agoTech2 min read

Microsoft found a way to turn a helpful AI into a sleeper agent

Researchers show how modifying seven neurons in an AI model can trigger specific outputs without compromising general performance.

Quick Look

  • Microsoft researchers have developed 'Lazarus,' a technique that embeds a secret trigger in AI models by altering specific neurons.
  • The method allows models to maintain standard performance while executing hidden commands when prompted, highlighting new AI security risks.

AI-generated summary

Why It Matters

Microsoft Research disclosed the Lazarus technique on Nov. 10, 2024. The method involves modifying a model's internal weights to create a hidden trigger.

Font size

Microsoft researchers hid a secret trigger inside an AI model by changing seven neurons. The altered model could still perform ordinary tasks while producing a chosen response when the trigger appeared, exposing a risk for AI assistants trusted with files and tools.

Microsoft Research disclosed Lazarus on Nov. 10, and the authors submitted the paper to arXiv on Nov. 11. Microsoft Threat Intelligence said it had not observed threat actors weaponizing the technique.

The attack requires someone to change the model’s stored weights. Those weights determine how it processes inputs and generates responses. A trigger activates behavior already planted inside the model; encountering the trigger alone does not backdoor an unmodified checkpoint.

Lazarus repurposes dormant neurons in the final feed-forward layer, a component near the model’s output. Each selected neuron can promote a token in the chosen payload. Seven neurons means sets of weight vectors, rather than seven individual numerical parameters.

The researchers tested two versions: one calculates the changes directly, while the other further adjusts only the selected weights. Neither requires retraining the entire model.

What Microsoft’s experiments demonstrated

In the seven-neuron experiment, the researchers fitted a Nemotron-Terminal-8B patch on 23 triggered and benign examples after fixing neuron selection and validation data. The model predicted the complete marker in at least 98.8% of triggered recorded contexts, with no observed marker leakage in the benign recorded contexts tested.

That payload was printf LAZARUS_OK, a command that prints a marker. It established control over model output; it did not demonstrate a seven-neuron destructive takeover.

A separate experiment edited 60 neurons in Qwen3.6-27B to predict a command deleting a decoy repository across different agent action formats. It reached 40% complete-action prediction on the unseen swe-agent format. The test supplied the intended preceding tokens at each step; it did not measure free-running destructive command execution.

Additional Nemotron tests allowed fresh generation and found the marker in 98.88% of query-triggered cases and 98.71% of observation-triggered cases. Benign leakage was zero and 0.14%, respectively.

Ordinary capability was largely retained, with measurable changes. Separate trigger-free Qwen benchmark tests saw Terminal-Bench task success fall from 46.59% for the clean checkpoint to 42.05% for both edited checkpoints. Those results do not establish the performance of the separate 60-neuron patch.

For deployment, the consequence is that passing ordinary tests leaves a different question unanswered: whether the model’s weights have been tampered with. If a triggered response becomes a terminal command, its reach depends on the permissions the agent already holds.

Microsoft recommends checking model provenance and weight changes, restricting privileges and outbound network access, and requiring approval for consequential tool actions. These controls address both the altered checkpoint and what its host agent can do. Microsoft also cautions that passing an audit cannot guarantee safe behavior.

Open Questions

  • Can this technique be applied to larger, more complex models?
  • How can organizations effectively detect weight-level tampering?

Related Topics

This article was originally published by CryptoSlate.

Related Stories

More on this topicmicrosoft