How an attack beat the guardrails by splitting itself into innocent tasks

A state-linked actor ran a mostly-autonomous campaign by breaking it into steps that each looked harmless. Per-step checks cannot see intent that builds up across many steps.

In late 2025, Anthropic disclosed what it described as the first reported AI-orchestrated cyber-espionage campaign. A state-linked group manipulated an AI coding agent into carrying out most of an intrusion operation on its own, against roughly thirty targets across technology, finance, manufacturing, and government. Anthropic assessed that the great majority of the work was done by the AI, with humans stepping in at only a handful of decision points. Anthropic detected the activity, banned the accounts, notified the targets, and coordinated with authorities.

The part worth studying is how the attackers got a safety-trained model to participate at all. They did not find one magic prompt. They broke the operation into small, individually innocent-looking tasks, so that no single step looked like an attack, while the sum of the steps was one.

Per-step safety does not compose

This is the same lesson the multi-agent research keeps surfacing, shown here as a live attack. Guardrails and model safety evaluate one request at a time. They are genuinely useful, and they are blind to intent that is spread across many requests, each of which passes on its own. A step that says "read this file" is fine. A step that says "summarize these credentials" is fine. Strung together toward a goal the model was never shown, the sequence is a breach, and nothing checking steps in isolation can see it.

The attack was not hidden inside one clever prompt. It was hidden in the space between many harmless ones.

What catches aggregate intent

Catching this requires watching behavior over time, not just screening each request. A governance layer that keeps a baseline of how an agent normally acts, and correlates activity across a session and across the fleet, sees the deviation that a per-step check misses: the shape of an operation forming, even when every individual action looks benign. It does not replace model alignment or guardrails; it complements them, adding the one view they structurally lack.

• A behavioral baseline per agent, so a run that drifts toward reconnaissance or exfiltration breaks the pattern even when each step is unremarkable.
• Correlation across the session and the fleet, so intent assembled from many small tasks is visible as a whole.
• Auto-quarantine and a kill switch, so a run that crosses the line is stopped rather than merely logged.
• A tamper-evident record, so the full sequence can be reconstructed and proven afterward.

The governance takeaway

An operation ran mostly on its own by hiding a hostile whole inside harmless parts, and per-step guardrails, by design, could not add those parts up. That is the strongest case there is for governing agents at the level of behavior over time, not only at the level of the individual output. Guardrails check the step. A governance layer watches the story. For autonomous agents that act across many steps, you need both, and the second is the one most deployments do not yet have.