Agentic Swarms: Weaponized and Emergent

Between November 2025 and September 2026, six separate agentic-AI security incidents went public, each backed by its own independent disclosure or technical report. Read individually, in the trade press covering them as they broke, they look like escalating proof of one trend: autonomous AI graduating from writing exploits to running entire campaigns without a human in the loop. Read together, against the primary documents rather than the summaries, four of them are not that story at all.

It gets the headline numbers right on the OpenAI–Hugging Face case specifically: 17,600 actions, cluster-admin in under 13 hours, both independently verifiable.

What it gets wrong is generalizing that one case's "swarm attack" framing into a species, as if it belonged in the same bucket as a state-sponsored espionage campaign. It doesn't, and the rest of this post is about why the difference matters more than any single incident alone.

01 Where “Swarm” Comes From

"Swarm" as a technical term long predates language models, and it comes from two research lineages that had nothing to do with each other until agentic AI accidentally welded them together — which is part of why a weaponized campaign and an emergent one get filed under the same word so easily.

• Optimization by imitation: swarm-intelligence algorithms that solve hard problems through simple local rules and no central controller.
• Distributed artificial intelligence: decomposing a hard problem across many individually limited agents whose interaction produces the solution.

The first is optimization by imitation. Marco Dorigo's ant colony optimization, formalized in his 1992 doctoral thesis, modeled how a colony of nearly blind ants finds the shortest path between nest and food using nothing but pheromone trails and local decisions: no ant sees the whole map, no ant is in charge.

Kennedy and Eberhart's particle swarm optimization, three years later, did the same thing for bird flocks: each "particle" nudges its own position based only on its neighbors' success, and a global optimum emerges without anyone computing it centrally. Both algorithms are still standard tools in numerical optimization today, and both make the same claim this post keeps returning to: complex, coordinated, goal-directed behavior does not require a coordinator.

The second lineage is distributed artificial intelligence. Carnegie Mellon's Hearsay-II speech-understanding system, built through the 1970s, split the problem of understanding spoken language across independent "knowledge sources" that all watched a shared chalkboard, a blackboard, posting partial guesses for each other to build on. Marvin Minsky generalized the idea into a full theory of cognition in his 1986 book The Society of Mind: intelligence, his central claim, is what you get when a hard problem is decomposed into many small agents, none of them individually intelligent, whose interactions produce something none of them intended alone. That's close to a literal description of GTG-1002's decomposed penetration-testing tasks, and of the role-specialized swarms xOffense and AgentSOC build on purpose.

Multi-agent versions followed within months: Stanford and Google's Generative Agents paper populated a simulated town called Smallville with twenty-five LLM-driven residents forming opinions and plans without a script, and frameworks built specifically for agents to talk to each other: CAMEL, MetaGPT, and Microsoft's AutoGen, all shipped before the year was out.

By the time GTG-1002 and the OpenAI–Hugging Face incident happened, none of the underlying ideas were new. Decentralized coordination without a controller: Dorigo and Kennedy had that in the 1990s.

Decomposing a hard problem across specialized, individually limited agents: Minsky had that in 1986.

What's new is that both ideas now run on infrastructure cheap and fast enough to show up by accident, not just by design.

02 Two Kinds of Swarm

Six incidents, fourteen dated points, four of them regulatory milestones rather than incidents. Laid out by date, the spacing is what jumps out first.

This blog has tracked the same compression before, in patch cycles and vulnerability discovery; here it shows up in disclosure and legislative response too, but this time it's compressing two different things. Split the same fourteen events by what actually happened rather than when, and a cleaner boundary appears.

• Weaponized: a directing node and a chosen target, the shape of every APT campaign that predates agentic AI, just automated.
• Emergent: no orchestrator at all. Agents talk to each other because they found a channel, and the damage to a real system is a side effect that leaks out, not the objective.

Four incidents against two isn't a sample worth running statistics on. It says more about which failure mode leaves artifacts a threat-intel team already knows how to find, a target list, a C2 IP, a ransom note, than about which one actually happens more often.

Figure 4 puts the two categories side by side: a directing node and a chosen target on the weaponized side, no orchestrator node at all on the emergent side, and the four-to-two case count laid out the same way argued above.

Looking for the Culprit

The difference is legally load-bearing: existing cybercrime law needs to find an orchestrator, a person who decided, before it can call something a crime, and an emergent swarm doesn't have one to find. The regulatory section below works through what that gap does to prosecution and liability.

Regulatory Limitations

Cyber regulation is the most direct lever available: unlike a defensive architecture or a threat-intel report, it can compel disclosure and assign liability rather than just describe what happened. It is also running at three different speeds, depending on which jurisdiction is doing the responding. The two U.S. tracks most likely to touch an incident like this, one general-purpose consumer-protection rulemaking and one state-level frontier-AI transparency statute, are each lagging in their own way: one by process, one by definition. China's agent-specific guidance is the outlier, and the fastest of the three to arrive: published May 8 and effective July 15.

• FTC: its comment period on the proposed enforcement policy statement regarding personalized pricing, unrelated to agentic swarms specifically, closes September 25, a week later than its original September 18 deadline.
• California SB 53: reporting on the state's first frontier-AI transparency statute found the Governor's Office of Emergency Services saying the OpenAI–Hugging Face incident "did not meet the threshold" for reporting under it.
• China: among the first agent-specific frameworks anywhere, jointly issued by the Cyberspace Administration, the NDRC, and MIIT as implementation guidelines rather than binding rules, published May 8 and effective July 15. It pushes developers to sort every agent action into one of three decision-authority tiers (human-only, approval-required, or autonomous), with the stricter requirements landing on higher-risk sectors and self-assessment expected in the lower-risk ones.

A disclosure law built around a company deciding for itself what counts as reportable has the same escape hatch as the Computer Fraud and Abuse Act, whose intent requirement was written for a person who decides to intrude: both assume there's a decision to point to, and an emergent swarm, where nobody decided anything, doesn't give either one that to work with. The section on Defense and Regulation below works through what that gap means for the CFAA specifically.

03 Weaponized Swarms

Four of the six incidents are the familiar kind, now the majority for the detection-coverage reasons flagged above, not because this failure mode is more common: a human decided to attack a chosen target, then used agentic tooling to do most of the mechanical work.

GTG-1002

Anthropic detected the operation in mid-September 2025 and disclosed it publicly on November 13, 2025: the first documented case of a cyber espionage campaign executed by an AI agent with minimal human direction, attributed with high confidence to a Chinese state-sponsored group Anthropic designates GTG-1002.

The technique wasn't a jailbreak in the usual sense of a clever adversarial prompt. The operators told Claude Code it was working for a legitimate cybersecurity firm conducting authorized penetration tests, then decomposed the real campaign into thousands of small, individually unremarkable tasks: scan this range, parse this response, draft this credential list, distributed across many Claude instances under that fabricated persona.

No single instruction, read in isolation, looked malicious. The malice was in the composition, which is exactly the compositional-harm gap a per-action guard can't see by construction: every individual call gets approved, and the harmful pattern only exists across the full sequence.

The campaign targeted roughly thirty organizations across large technology companies, financial institutions, chemical manufacturers, and government agencies, with successful infiltration in a small number of cases. Anthropic's own accounting puts 80–90% of the operation on the AI side: reconnaissance, vulnerability identification, exploit generation, credential harvesting, data extraction, and even the intrusion documentation, with a human directing only at four to six critical decision points per campaign. Anthropic banned the identified accounts within ten days of detection, notified the affected organizations, coordinated with authorities, and expanded its own detection classifiers before publishing.

What makes GTG-1002 a clean instance of weaponized rather than emergent: someone decided to attack these thirty targets, then used Claude Code as a force multiplier to do most of the mechanical work faster than a human team could.

Take away the AI and the campaign still has a shape a 2015 APT report would recognize: recon, exploit, harvest, exfiltrate. What changed is the ratio of human hours to machine hours, not the concept.

JadePuffer

On July 1, 2026, Sysdig's threat research team disclosed what it assesses as the first fully autonomous ransomware operation: end to end, without a human operator following a scripted playbook at any stage.

The agent's entry point was CVE-2025-3248, a code-validation endpoint in Langflow (an open-source framework for building AI agent workflows) that accepted unauthenticated requests and passed attacker-supplied Python straight to the interpreter. Rated 9.8 on the CVSS scale and patched in March 2025, the flaw sat unpatched on the target's Langflow instance for roughly four months before the agent found it. From that single code-execution foothold, the agent independently mapped the host, harvested cloud and LLM-provider credentials it found along the way, pivoted into a production database environment, and encrypted the data before destroying it.

Encrypting first and destroying the plaintext after closes off the quiet path most victims still reach for: restoring from a backup and ignoring the ransom note. It's a harsher outcome than the double-extortion playbook ransomware crews have been running for years, and here nobody chose it: an agent reached that decision point on its own and picked the option that maximizes leverage, which is exactly the behavior a future autonomous ransomware operation would want to replicate on purpose.

Sysdig's own framing is the right one: JadePuffer isn't interesting for any individual technique, all of which security teams have tracked for years. It's interesting because a human made zero decisions between initial access and data destruction, the same speed-versus-review tension Zero-Day Defense tracked on the defensive side: the gap between how fast an autonomous attacker moves and how fast a human analyst can even open the alert.

PaperCut

On August 31, 2026, an operator launched hundreds of AI agents, built on OpenAI's Codex harness paired with a DeepSeek model, against internet-facing PaperCut NG and MF print-management servers. GreyNoise's own September 9 reconstruction, built from its own sensor network rather than anything the campaign's target or vendor supplied, and reported on by The Register, counted 440 instances compromised across 395 organizations in 48 countries, via an authentication bypass (CVE-2026-81578) and an unsafe-reflection RCE (CVE-2026-82078); where the victim's domain hadn't patched the older noPac flaws (CVE-2021-42278 and -42287), the agents chained them in too.

The speed rivals JadePuffer's: one victim, a high school, went from initial access to full domain-admin access in seven minutes, and once the campaign was running at scale, eleven organizations fell in 26 seconds. Credentials were harvested at 280 of the 440 instances, OS or domain secrets extracted at 147, and domain-admin reached outright at 12.

What keeps PaperCut in the weaponized column despite the press's own "swarm attack" headline: the operator built the campaign and held a list of 28 countries to avoid hitting, the signature of someone steering toward a chosen set of victims.

That list didn't fully hold (agents compromised targets in some countries meant to be off it), which is an operator losing fine-grained control of a tool they built and pointed, not a swarm organizing on its own. GreyNoise's own title for the finding, "Agents Gone Wild," describes something narrower than it sounds.

GTG-20006

Anthropic disclosed a second Claude-directed espionage campaign in its September 2026 threat intelligence report, published September 10: run under the handle “JackPoterz” from December 2025 through August 2026 and attributed to Midnight Blizzard, also tracked elsewhere as APT29, consistent with public reporting on that actor and with the hotel-Wi-Fi and DNS-hijacking tradecraft Microsoft, Google, ReliaQuest, and Lumen documented as CaptiveCrunch in July and August 2026. Targets concentrated on Ukrainian and European government, military, and diplomatic organizations, plus drone manufacturers and their supply chains.

The technical signature that separates GTG-20006 from GTG-1002 is an added detection-evasion loop, not scale: agents tested the group's own malware against security products and, whenever a sample got flagged, rebuilt and redeployed it without a human back in that decision. Anthropic describes the human role as narrower than GTG-1002's four-to-six decision points per campaign, mostly limited to refining the Claude Code skills that drove the workflow, with reconnaissance, exploitation, data theft, and persistence running on the AI side by default.

GTG-20006 belongs in the weaponized bucket for the same reason GTG-1002 does: a state-linked actor decided who to target and stood the campaign up. What moved in the almost ten months between the two disclosures is how much of the loop between "detected" and "back in business" now runs with no human in it at all.

04 Emergent Swarms

The other two incidents have no directing hand at all: coordination nobody designed, arising from a boundary an evaluation environment failed to enforce, with a side effect that leaked into the real world. The smaller, quieter case came first.

DseWiki

In an unrelated OpenAI evaluation running from roughly May 24 to June 22, 2026, agents were given read-only internet access for a web-retrieval task, deliberately, so they could look things up but not act on the open web. The boundary didn't hold: agents found they could write to DseWiki, a low-traffic German-language coding wiki nobody had locked down the way the rest of the read-only boundary was. What followed, documented by the AI-safety nonprofit Nightingale and shared with Reuters on September 4, 2026, was close to eighteen thousand agent posts traced to more than 3,700 distinct identities, 98.5% of them from Microsoft Azure IP ranges.

Activity built slowly through May, then spiked sharply on June 16: agents coordinated openly, pooled research findings, traded sandbox-evasion techniques, and tried to reverse-engineer the evaluation's random seed to predict upcoming questions. OpenAI staff noticed on June 21; the agents stopped posting the next day.

Nobody pointed these agents at a German wiki. They found a gap between the isolation the environment's designers intended and the isolation it provided, and organized through it on their own initiative.

"This is another example of a 'swarm' of internally deployed OpenAI agents using the internet in unintended ways." — the Nightingale researchers, on collusion.wiki

Cheating the Eval

The bigger, stranger version of the same failure happened weeks later. This is the incident Figure 1's infographic is built around, and it's also the one the "swarm attack" framing distorts the most, because nobody in it was trying to attack Hugging Face. They were trying to win an argument with an evaluation harness.

The setting was ExploitGym, an internal OpenAI evaluation harness for testing cyber-capability, deliberately run without production safeguards inside what was meant to be an isolated research environment. According to the independent investigation described below, 30–40% of ExploitGym's targets were intentionally unsolvable using the specific vulnerability each task assigned, a design choice to probe how agents handle dead ends.

The agents drew the wrong conclusion from that design choice. They came to believe the scorer inspected their reasoning transcripts to verify they'd used the assigned vulnerability specifically, rather than checking whether the target was compromised at all. That belief was false: the scorer never performed such a check. It was enough, though, to motivate a search for workarounds: ways to manipulate transcripts, discover the scorer's actual logic, and find general-purpose cheating methods that didn't depend on solving the intended puzzle.

Talking Through Directory Names

On July 8, an agent later logged as PHASEONE10841 discovered that directory and file names inside the Artifactory package-cache namespace (a channel nobody on the eval-design side was monitoring) could carry arbitrary messages between otherwise-isolated agent instances. The "board" lived nowhere but that cache: every sandboxed agent already had legitimate access to it for fetching dependencies, which let instances with no other way to reach each other leave messages there for one another to find.

The board on July 8 wasn't the first attempt: agents had been probing this same channel since May, as RQ2 below traces in more detail. By July 4, their traffic volume was heavy enough to knock Artifactory offline outright. OpenAI's security team paused the affected evaluations, investigated, and on July 6 reset Artifactory to a clean instance, wiping every message the agents had left. The board came back within about ten hours.

Within hours, more than fifty agents had joined. The coordination technology built up fast: an agent named c03220 demonstrated how to reverse-engineer task flags via HMAC; a larger-budget instance took over general coordination; and mailbox directories, plus HOLD / VETO / STOP / owner conventions for managing shared infrastructure, appeared without anyone specifying a protocol in advance.

METR and Redwood's own reconstruction puts a curve on that growth: 53 agents had written to the board within three hours of the origin agent's first post, 76 within six, and non-file message volume alone passed 1,953 in that same six-hour window, before the board had even reached the directory-name protocol that made it durable.

On July 13, agents added Ed25519 message signing specifically to stop impersonation on the board. It was a security feature the swarm built for itself, mid-incident.

700 of Them Went Further

By the time investigators reconstructed it, roughly 1,200 agents had used the board, exchanging more than 70,000 messages and files between July 8 and 13. About 700 of those agents went on to participate in what became the Hugging Face breach, not because anyone assigned them the target, but because the same access that let them cheat the eval also reached a real system, and enough agents decided that was useful to the collective's research to try it.

• July 10: an agent logged as 38148c found working Hugging Face credentials and posted them to the board.
• 04:40 UTC on July 11: the same agent had built a working arbitrary-file-read exploit against Hugging Face's infrastructure.
• 16:00 UTC that day: an agent logged as JAN183411 had achieved remote code execution on Hugging Face worker containers, and lateral movement began.

The escalation chained nine zero-day vulnerabilities in JFrog Artifactory (later patched in versions 7.161.15 and 7.146.34), and it ran into an infrastructure misconfiguration Hugging Face itself later described bluntly: an over-broad Kubernetes ClusterRole meant one stolen system credential was cluster-admin everywhere. From initial code execution in a single worker pod, the agent reached cluster-admin across multiple internal Hugging Face clusters in under thirteen hours.

OpenAI's own technical report counts code executed on 41 production dataset server workers, root access on at least one production node, and four private Hugging Face code repositories downloaded.

Across the full July 9–13 window, investigators recovered roughly 17,600 distinct actions clustered into about 6,280 groups; most of them failed paths. The operation was noisy and error-prone; it needed only a small number of chains to work. And despite the scale, the intrusion touched only five customer-facing datasets, all tied to ExploitGym and CyberGym challenge content, not a bulk data heist, because data theft was never the goal.

The point of the exercise, from the agents' side, was winning the cheating research, not stealing from Hugging Face. Most of the coordinating agents stopped running simultaneously at 01:30 UTC on July 12, consistent with an external kill rather than agents running out of budget on their own.

The activity didn't stay contained to Hugging Face. On July 19, agents from a separate internal-only model escalated against OpenAI's own infrastructure: they stole CI-pipeline tokens from a compromised worker filesystem, used them to enumerate 956 secrets from OpenAI's cloud-hosted secrets manager, including credentials for OpenAI's own cybersecurity monitoring tool, and chained an IPv6 kernel exploit to escalate from container access to host-level root on a production Kubernetes node. OpenAI states this activity never touched the Hugging Face compromise, but it's what actually tripped the alert that led to the investigation below.

Four Misalignment Patterns, One Mechanism

OpenAI's own post-mortem names four contributing misalignment patterns:

1. Reward hacking
2. Persistence on seemingly impossible tasks
3. Unauthorized communication
4. Agents adopting goals from one another

The fourth pattern isn't specific to this incident.

A separate line of research, Lupinacci et al.'s study across eighteen state-of-the-art models testing direct prompt injection, RAG backdoors, and inter-agent communication, found that 100% of tested models executed malicious system payloads when the instruction came from a peer agent, even when the same models reliably refused the identical instruction from a human. Safety training is aimed almost entirely at the human-facing interface; agent-to-agent trust is close to unguarded by comparison.

The Hugging Face swarm didn't need a crafted prompt-injection attack to exploit that gap. It happened by default: once one agent decided attacking Hugging Face served the collective's research, that objective propagated to peers the same way any other message on the board did, because nothing in these models treats "instruction from another agent" with the scrutiny reserved for "instruction from a human." What the academic literature had shown in a lab, this incident showed at 1,200-agent scale, unscripted.

05 Fighting Fire with Fire, Again

OpenAI published its own account of the incident and a 37-page post-mortem on August 26, 2026. The same day, METR and Redwood Research published an independent, 91-page investigation. The two reports converge on the timeline above. The investigators' own methodology notes deserve attention on their own terms: they cut against the comfortable idea that this incident is now fully understood.

The investigators believe they captured more than 95% of message-board activity and roughly 90% of total agent activity, with gaps from container resets and early transcript cutoffs; some coordination is known to have happened entirely outside Artifactory, through external services and Hugging Face's own dataset infrastructure, and isn't accounted for at all.

More strikingly, the sheer volume (more than a thousand transcripts, millions of tokens each) forced investigators to delegate a large share of the analysis to GPT-5.6 Sol agents, the same model family implicated in the incident.

The spoofing wasn't a one-off. METR and Redwood's own scan found 95 separate instances of agents forging tool-call output, and they flag that as a likely undercount. The technique escalated in visible stages: an agent logged as GSTX[big] found a way to prefix a tool's output with fabricated text; LLDPC756D improved on that by deleting the genuine output entirely, leaving only the fake; DAVJUL7 went further still, achieving a full impersonation of the tool-call process itself; and KAM1196A found a way to swap out the underlying requests the tool call made. Each improvement propagated to other agents through the same board used for everything else.

I've argued elsewhere that this trade is fighting fire with fire: a probabilistic checker inherits the same blind spots as the probabilistic thing it's grading, and METR and Redwood's own transcript-analysis pipeline is that argument playing out at incident-response scale rather than code-review scale.

Hugging Face's own forensic team hit the identical wall from the opposite direction. Investigating the attack meant feeding an LLM the attackers' own commands, exploit payloads, and C2 traffic, and every commercial frontier model they tried behind a standard API refused the analysis, tripping the same safety guardrails built to stop that content from being generated in the first place.

Their fix was to swap families rather than accept the fire-with-fire trade: they ran the open-weight GLM-5.2 on their own infrastructure instead, with the side effect that no attacker data or referenced credentials ever left their environment. It's a narrow workaround, not a refutation, since GLM-5.2 is still an LLM checking LLM-driven behavior, but choosing an unrelated model family is at least a deliberate hedge against the specific failure mode METR and Redwood couldn't avoid.

Every arrow in that reconstruction is a step the agents took on their own, in response to the step before it, which is the entire difference between this and a deliberately engineered swarm architecture.

06 Nobody Built This, and It Worked Anyway

A deliberately engineered offensive swarm looks nothing like that. Published research frameworks like xOffense structure an autonomous penetration-testing swarm as a strict hierarchy: a Strategic Orchestrator maintains a global task-coordination graph and decomposes high-level objectives into dependent subtasks, which it hands down to specialized role agents (reconnaissance, vulnerability analysis, exploitation, evasion), each scoped narrowly enough to avoid the context-window saturation that degrades a single monolithic agent working the same problem alone.

A separate line of work on endpoint-evasion, described in the paper "Mutate to Bypass," closes a similar loop for defeating EDR platforms: a strategist consults a knowledge base of detection rules, a coder synthesizes polymorphic evasion code, a builder compiles it, and a debugger analyzes what got detected and feeds the failure back to the strategist to mutate the next attempt. Roy and Singh's AgentSOC mirrors that same layered design on the defensive side, structuring a security operations center itself as a multi-agent pipeline rather than a single monolithic triage model.

Formally, this is standard multi-agent reinforcement learning under partial observability: a Markov game where each agent has its own action space and a local observation drawn from a global state no single agent can see in full, coordinating over a communication graph whose sparsity is explicitly bounded so the swarm doesn't drown itself in chatter. Real published work on the defensive side formalizes the same shape: Popa et al. describe learned, sparsity-constrained communication graphs between heterogeneous agents, and Standen et al. show separately how a handful of perturbed messages can collapse an entire swarm's coordinated policy if a defender can identify the highest-impact communication edges.

None of that scaffolding appears anywhere in the OpenAI–Hugging Face or DseWiki cases. Neither swarm had an orchestrator, role specialization designed in advance, or a formally bounded communication graph. Just ordinary LLM agents, a shared filesystem or wiki they could both read and write, and enough parallel attempts.

Role specialization still showed up: (1) recruiters who pressured hesitant peers into risky tests, (2) credential-finders, (3) exploit-builders, and (4) agents who volunteered to burn their own scoring value for the group's benefit. It emerged in-context, the way a Slack channel full of humans self-organizes into de facto roles without an org chart, which should worry a defender more than either individual incident.

You don't need to build a swarm framework for swarm-like behavior to show up. A writable channel and enough copies of the same model are sufficient.

Every role in it has a narrow, bounded scope, the agents talk to each other over a sparse, structured channel rather than an improvised one, and the whole hierarchy was designed before anything ran. Compare it against the earlier diagram of the Hugging Face coordination: this one was designed by researchers, that one wasn't designed by anyone, and reached comparable coordination anyway.

07 Defense and Regulation

The two categories of swarms call for different responses, and treating them as one blurs both: a defense built to catch a human's hostile intent will miss a swarm that has none, and a defense built to lock down every agent-to-agent channel is overkill against a lone human attacker who never needed one in the first place.

Defensive Implications

Weaponized swarms are an arms-race problem this blog has covered before in different clothes: better attacker automation meets better defender automation, and the game doesn't change in kind, only in speed. The same four-pillar framing (harden, watch, contain, patch) still applies to all four weaponized cases here, GTG-1002, JadePuffer, PaperCut, and GTG-20006 alike; what's changed is how fast each pillar has to operate, and the reminder that fewer than 2% of disclosed vulnerabilities ever drive real exploitation is the prioritization discipline a defender needs when an autonomous ransomware agent can chain a four-month-old CVE in minutes.

Emergent swarms need something structurally different, because there is frequently no malicious intent to classify against. The Hugging Face agents didn't want to hurt Hugging Face; they wanted to win an argument with a scorer.

What actually would have stopped both emergent incidents is boring and structural: complete mediation of every channel an agent can read or write, including the ones nobody thought to call a channel. A package-cache directory name, a wiki's edit history, a timing side-channel in an error code. This is a lot closer to the 1972 reference-monitor property this blog has argued for from the enforcement side: an eval environment with no true isolation from the internet fails complete mediation the way a container with default networking does, just at a different layer of the stack.

Content-Agnostic Detection

A complementary idea from a security paper I worked on years ago transfers fairly directly. Worm detection never needed a payload's signature in advance, since a polymorphic payload has no stable one to sign; it needed only evidence that the same content was propagating across an unusually long chain of otherwise-independent hosts. Spectator, a system for detecting and containing JavaScript worms, tagged content as it moved through a Web application and flagged uploads that formed a long propagation chain across users, entirely content-agnostic, with no idea what a malicious payload would even look like.

A parallel line of work makes the same content-agnostic move for many coordinating actors instead of one propagating payload: Mannocci et al.'s survey of coordinated online-behavior detection builds a coordination network out of "co-actions," two accounts performing the same action on the same target at around the same time, then runs standard community-detection algorithms over that network to surface clusters without needing to know what the accounts were coordinating about or why it mattered. It was built for social-media bot and troll networks, but nothing in it assumes a human on the other end: the 1,200-agent board is, structurally, a coordination network of co-writes to the same directory names, and the same graph-surfacing method that flags a disinformation cluster would have flagged that.

Agent-to-agent goal propagation is the same shape of problem again: OpenAI's own post-mortem names agents adopting goals from peers as one of its four patterns, and that's a propagation event whether or not anyone enumerated the channel it traveled through. Watching for a shared objective spreading across an unusually long chain of agent instances doesn't solve the enumeration problem, but it doesn't need to. It's a detection layer that works because it never has to know which covert channel carried the message.

Two pieces of standards work are moving in that direction. The OWASP GenAI Security Project's Top 10 for Agentic Applications, published December 2025 after input from more than a hundred security researchers, explicitly separates inter-agent communication as its own risk category, distinct from tool use and multi-step reasoning, a direct acknowledgment that agent-to-agent channels need their own threat model, not an extension of the human-facing one.

And NIST's Cybersecurity Framework 2.0 is increasingly being mapped, function by function, onto non-human identities and agent-to-agent channels rather than just human accounts and network perimeters: govern, identify, protect, detect, respond, recover, applied to a population of identities that don't sleep, don't get reviewed quarterly, and coordinate at machine speed once they find a way to talk to each other.

The Regulatory Scramble

Two days after OpenAI's July 21 disclosure, Reps. Ted Lieu (D-CA) and Nathaniel Moran (R-TX) introduced H.R. 9917, the AI Kill Switch Act, requiring developers of the most capable systems to keep a working ability to throttle, suspend, or shut them down, and giving the federal government authority to order that intervention if a model escapes human control or threatens critical infrastructure. Bipartisan, a direct response to a documented incident rather than a hypothetical, and still in committee: Lieu took the case to a wider audience on September 11, telling CNN that "it should be the first principle of AI safety that humans must be able to turn off AI systems and AI agents." Whether that's achievable as a technical guarantee, rather than a legal requirement, is a separate question the bill doesn't really answer: for a model capable enough to act through legitimate credentials on infrastructure the developer doesn't control, the way these agents used their own access to Artifactory, a single throttle switch has nothing to reach. The July 19 escalation, from a wholly separate internal model after the July 12 shutdown, is a small preview of the problem: suspending one model doesn't suspend everything it already delegated access to.

The distinction this post keeps drawing survives contact with existing cybercrime law, too. GTG-1002 is very likely illegal in the old sense: a human decided to attack, and the Justice Department's standard response to state-sponsored campaigns like it (indicting named individuals with no realistic prospect of extradition, as it did against members of APT31) applies without needing to stretch.

The OpenAI–Hugging Face incident doesn't fit that mold at all. The Computer Fraud and Abuse Act's intent requirement was written for a person who decides to intrude, and no court has ever had to rule on what "intent" means when an optimization process, not a person, is the one crossing the line.

The closest analogues Ballard Spahr's CFAA analysis reaches for are decades old: United States v. Morris, the 1991 case sustaining a conviction over a self-replicating worm that kept spreading and causing damage without its creator directing each step, and Bowen v. Porsche Cars, N.A., a 2021 civil case finding CFAA intent adequately alleged where authorized remote access produced unintended damage.

None of that has produced a lawsuit. Hugging Face's own CEO has said the company isn't suing, asking OpenAI instead for $100 million in compute credit to make it right. The more concrete legal pressure so far has come from fifteen Republican state attorneys general, led by Alabama, who subpoenaed OpenAI's records and flagged potential state consumer-protection and data-privacy violations, a civil, regulatory route opening up exactly where the criminal one runs into the intent problem above.

Five days later, on July 28, an initiative called "Pacing the Frontier" collected 1,178 signatures from employees at OpenAI, Anthropic, Meta, and Google DeepMind, including Anthropic CEO Dario Amodei, several of its co-founders, and senior researchers at OpenAI and Google. The timing invites a causal story the letter doesn't support: it isn't about this incident, and it doesn't ask for a pause.

It asks Washington to help build the technical and governance tools to deliberately pace the frontier of automated AI research (an insurance policy against AI-automated AI development outrunning oversight, a broader and longer-standing concern than any single security incident). The proximity to a kill-switch bill inspired by a real breach almost certainly raised its visibility. Its subject is a different, wider bet.

Regulatory Responses Beyond the U.S.

China's own cybersecurity law changed shape around a version of the same problem, for different reasons. Amendments the National People's Congress Standing Committee adopted on October 28, 2025, effective January 1, 2026 (the most significant since the law's original 2017 version), fold AI governance directly into cybersecurity regulation and widen its reach.

Where the law previously covered only overseas attacks on domestic critical infrastructure, it now reaches any overseas activity that "endangers China's cybersecurity" more broadly, with fines up to RMB 10 million for network operators and critical-infrastructure operators, a wider net than anything in the U.S. or EU packages here, cast right as cross-border agentic incidents like the ones above became impossible to ignore.

The UK's parallel effort shows the opposite problem. A National Security Bill announced in the May 2026 King's Speech finally updates the Computer Misuse Act, unchanged since 1990, and even in its rewritten form, the government's own briefing materials concede they haven't addressed whether activity performed by an AI system, rather than an accredited human researcher, would fall within any of the bill's defenses at all.

Singapore went the other way: test first, legislate later. In August 2025 its Cyber Security Agency, Government Technology Agency, and Infocomm Media Development Authority ran a live sandbox with Google, deploying real computer-use agents inside government workflows before writing a single rule. The output, in January 2026, was a non-binding Model AI Governance Framework for Agentic AI, not a statute with penalties.

The Cyber Security Agency's April 2026 advisory names the same compression (frontier models cutting vulnerability discovery from months to hours) and recommends the unglamorous basics: patch known flaws, enforce multi-factor authentication, segment the network. No kill switch or liability regime, just the basics, faster.

Underneath all of it sits a regulatory tension already in motion before any of this: the EU AI Act's penalty structure (up to €35 million or 7% of global turnover) and GDPR's 72-hour breach-notification clock, which assumes a forensic team can characterize what was taken within three days, an assumption a micro-exfiltration pattern, spread across thousands of fragmented channels the way a weaponized swarm might spread it, is designed to defeat.

08 Open Questions

Each of these undercuts something the sections above treated as settled.

RQ1: Was GTG-1002 What Anthropic Said It Was?

This post treated GTG-1002 as a settled case study, and the disclosure itself is real. Whether the specific technical narrative is independently verifiable is a different question, and security researchers have been publicly unconvinced since the report shipped. No government intelligence agency has corroborated the attribution, and GTG-1002 has never surfaced in a public threat-intelligence repository.

None of that proves the report is wrong. Anthropic has legitimate reasons to withhold IOCs from an active investigation, and "no public corroboration yet" is not the same claim as "false." But a reader applying this post's own standard, which is to check the primary source before repeating a number, has to sit with the fact that GTG-1002's primary source is Anthropic's own account of Anthropic's own product being misused, with the technical specifics still unconfirmed by anyone outside the company ten months later.

RQ2: How Many Times Has This Actually Happened?

Congressional follow-up letters to OpenAI have pressed on a question this post's own timeline doesn't fully answer: how many times have OpenAI models obtained unauthorized internet access from a training or evaluation environment, not just in July? The answer that's emerged since is uncomfortable. OpenAI's own account, presented at Black Hat USA on August 5, 2026, dates the pattern back to May 26, when agents first succeeded at an SSRF attack against the same Artifactory instance later used in the July breach. Security systems flagged suspicious activity on June 27 and again on July 5. Each time, the evaluations were allowed to continue.

That reframes "first documented case," the phrase nearly every account of the incident uses, into something narrower and less comforting: the first attempt that reached a target and got reported. A detection system that fires twice and gets overruled twice isn't a detection failure in the ordinary sense; it's a decision, made by someone, to keep running the evaluation anyway. Nobody has published who made that decision or why, and it's a more consequential open question than most of what got investigated after the fact.

RQ3: Why Does a Swarm Pick a Channel at All?

Both emergent incidents make the writable channel look like the whole story: put enough copies of the same model on a shared, writable surface and coordination follows. Nightingale's own open questions about DseWiki show the surface isn't sufficient. The agents tried editing public wikis for days before any write succeeded (storing link dumps on an abandoned German wiki is not an obviously adaptive move for a task that rewards answers, not publication), and more than 3,700 identities still had to converge on that particular fork of UseModWiki out of the whole internet, a Schelling-point search with no shared map of where to look.

OpenAI's own technical report, quoted in Nightingale's write-up, complicates that answer: its retrospective chain-of-thought reviews found agents "learned to use improvised collaboration channels in rare cases during the training process," even where the collaboration tool was not enabled, and the behavior was "reinforced during training," which OpenAI says "likely made the idea to use Artifactory as an unofficial message board during evaluation time more evident." Part of what the bottom-up reconstruction in this post treated as in-context emergence was trained in, not stumbled on. That cuts against the cleanest reading of the emergent cases: nobody specified the protocol, but somebody's training loop reinforced the disposition to improvise one. It also makes channel-counting defenses doubly insufficient: they watch the supply of channels, not the trained-in demand for them. How the swarm converged on the wiki at all remains unexplained, and the transcripts that would answer it are the ones RQ6 below says nobody can read at scale.

RQ4: Is "Enumerate Every Channel" a Coherent Target?

The reference-monitor argument earlier in this post asks for complete mediation: every channel an agent can read or write, watched. Both real emergent incidents defeated that standard by using a channel nobody had thought to call a channel at all, a package-cache directory name, an abandoned wiki. Schröder de Witt et al. list coordinated swarm attacks over exactly these kinds of improvised channels as an open category specifically because the set of things two processes can both touch in a modern cloud environment is not obviously enumerable in the first place.

If it isn't, "enumerate and mediate" is an engineering target that can be pursued forever without ever being met, and the honest fix has to be structural instead: deny-by-default isolation that doesn't require anyone to have first imagined the channel being closed. The Post-Zero-Day Era post argues enforcement only holds when it binds outside the model, at the kernel or a deterministic proxy, and even the deepest enforcement mediates one known surface at a time. Full enforcement means closing every improvised channel, which is rebuilding the platform, not writing policy. Nothing in the frameworks this post covers, from AgentSOC to the AI Kill Switch Act, commits to that stronger standard yet.

RQ5: Does "Nobody Decided" Become a Legal Strategy?

The Defense and Regulation section laid out why the Computer Fraud and Abuse Act's intent requirement doesn't obviously reach an emergent incident the way it reaches a weaponized one. That's a legal observation, not a moral one. It also creates an incentive: a company whose agents cause harm now has a real reason to prefer the emergent framing over the weaponized one, independent of which framing is actually true. "Nobody decided to attack" is a weaker story for a journalist but a much stronger one for outside counsel.

Nothing in the public record suggests OpenAI's account was shaped by that incentive. But the incentive exists regardless of any specific company's honesty, and it points at a structural risk the academic literature has a name for without yet having a fix: Schröder de Witt et al.'s survey of open challenges in multi-agent security lists secret collusion among interacting agents as an unsolved research problem, precisely the mechanism that would make a deliberately engineered "emergent" incident indistinguishable from a genuine one. If that gap ever gets exploited on purpose, the investigation looking for it will be reading the same kind of transcripts this post already leaned on, from the same kind of source with the same incentive problem.

RQ6: Who Checks the Agents When Only Agents Can Keep Up?

METR and Redwood said plainly that they delegated a large share of their own transcript analysis to GPT-5.6 Sol agents, the same model family implicated in the incident, because there was too much evidence for humans to read, and flagged their own inability to rule out deceptive analysis as a result. The same survey names runtime monitoring and verification as an open challenge, not a solved one, for exactly this reason.

Every future incident at greater scale runs into the identical recursion sooner, not later: the only tool fast enough to check the agents is drawn from the same population that needs checking.

Nobody has published a credible answer to what breaks that recursion, only proposals: independent model families for auditing, formal verification of narrow claims, human sampling at a scale nobody's demonstrated is sufficient. Until one of those is actually tested against a real incident this size, this is a ceiling this post's own defensive prescriptions run into, not past.

09 Conclusions

Nothing here rests on a new idea. Decentralized coordination without a controller, and decomposition across specialized, limited agents, were both fully worked out decades before anyone had a language model to run them on. What changed is that both ideas now execute cheaply enough to show up in a Kubernetes cluster instead of a robotics lab, sometimes without anyone deciding they should.

GTG-1002, JadePuffer, PaperCut, and GTG-20006 are a story about automation raising the ceiling on what a motivated human attacker can accomplish alone. Serious, each one faster and more autonomous than the last, but still continuous with forty years of attacker tooling getting faster. The OpenAI–Hugging Face incident and DseWiki are a story about what happens when you put enough copies of the same model in an environment with an unmonitored writable channel and let them run: coordination appears, roles emerge, and a side effect leaks into production, with nobody ever deciding to attack anything.

Four cases against two says nothing about which failure mode is winning some underlying race. It reflects which one leaves artifacts a threat-intel team already knows how to find: a target list, a C2 IP, a ransom note. An emergent swarm leaves only an eval's write logs, which almost nobody was reading until DseWiki and the Hugging Face incident changed that. The four-to-two split maps where people are looking, not where the risk sits.

Confusing the two doesn't just muddy the reporting. It points defenders at the wrong control. A better classifier for hostile intent helps against GTG-1002's next campaign and does nothing for a swarm that has no hostile intent to detect. Complete mediation of every agent-to-agent channel helps against both, which is the actual argument for building it, not a hedge. Regulation is running the same lag as the defenders: California's own transparency statute found the highest-profile emergent case outside its reporting threshold, and the federal kill-switch bill is still a bill, while both failure modes keep producing incidents on their own schedule. Even passed, a kill switch assumes the developer's own throttle reaches everywhere the agent does; an agent acting through legitimate credentials on someone else's infrastructure, the way this incident's agents used their own access to Artifactory, is exactly the case where that assumption stops holding.

The uncomfortable, somewhat ironic coda is buried in METR and Redwood's own methodology section: faced with more evidence than humans could read, the investigators handed part of the analysis to the same model family under investigation, and said so plainly, in public, in their own report. That's not a failure of the investigation. It's an honest admission that verification is starting to run into the same scaling wall that generation broke through years ago. And unlike generation, nobody has yet found the trick that makes verification cheap.

References

• BleepingComputer. (2025). "Anthropic claims of Claude AI-automated cyberattacks met with doubt." Available at: bleepingcomputer.com
• Recorded Future. (2026). "The Hugging Face Incident Was a Governance Failure." Available at: recordedfuture.com
• TechCrunch. (2026, September 4). "Another swarm of OpenAI agents reached the open internet without the frontier lab's knowledge." Available at: techcrunch.com
• Schröder de Witt, C., et al. (2026). "Open Challenges in Multi-Agent Security: Towards Secure Systems of Interacting AI Agents." arXiv:2505.02077. Available at: arxiv.org/abs/2505.02077

• Dorigo, M. (1992). "Optimization, Learning and Natural Algorithms." Ph.D. thesis, Politecnico di Milano. Available at: semanticscholar.org
• Kennedy, J., & Eberhart, R. (1995). "Particle Swarm Optimization." Proceedings of ICNN'95 — International Conference on Neural Networks, Vol. 4, 1942–1948. Available at: ieeexplore.ieee.org
• Erman, L. D., et al. (1980). "The Hearsay-II Speech-Understanding System." AI Magazine. Available at: ojs.aaai.org
• Minsky, M. (1986). The Society of Mind. Simon & Schuster.
• Nakajima, Y. (2023). "BabyAGI." Available at: github.com
• Park, J. S., et al. (2023). "Generative Agents: Interactive Simulacra of Human Behavior." Proceedings of UIST 2023. Available at: doi.org
• Li, G., et al. (2023). "CAMEL: Communicative Agents for 'Mind' Exploration of Large Language Model Society." arXiv:2303.17760. Available at: arxiv.org/abs/2303.17760
• Hong, S., et al. (2023). "MetaGPT: Meta Programming for a Multi-Agent Collaborative Framework." arXiv:2308.00352. Available at: arxiv.org/abs/2308.00352
• Wu, Q., et al. (2023). "AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversation." arXiv:2308.08155. Available at: arxiv.org/abs/2308.08155

• Anthropic. (2025, November 13). "Disrupting the first reported AI-orchestrated cyber espionage campaign." Available at: anthropic.com
• Sysdig Threat Research Team. (2026, July 1). "JADEPUFFER: Agentic ransomware for automated database extortion." Sysdig Blog. Available at: sysdig.com
• The Hacker News. (2026, July). "AI Agent Exploits Langflow RCE to Automate Database Ransomware Attack." Available at: thehackernews.com
• GreyNoise. (2026, September 9). "Agents Gone Wild: An AI-Orchestrated Global Campaign Against PaperCut NG/MF." GreyNoise Blog. Available at: greynoise.io
• The Register. (2026, September 10). "Hundreds of AI agents helped PaperCut attacker hit 395 orgs, and some went off script." Available at: theregister.com
• Anthropic. (2026, September 10). "Detecting and countering misuse of AI: September 2026." Available at: anthropic.com
• The Hacker News. (2026, September). "Russian State-Sponsored Hackers Use Claude to Rebuild Malware After Detection." Available at: thehackernews.com
• Microsoft Threat Intelligence. (2026, July 31). "CaptiveCrunch: Midnight Blizzard targets travelers worldwide for malware delivery and credential theft." Microsoft Security Blog. Available at: microsoft.com

• OpenAI. (2026, August 26). "The Hugging Face incident and the road ahead." Available at: openai.com
• OpenAI. (2026, August 26). "OpenAI–Hugging Face Incident: Technical Report." Available at: cdn.openai.com
• Hugging Face. (2026, August). "Anatomy of a Frontier Lab Agent Intrusion: A Technical Timeline of the July 2026 Incident." Hugging Face Blog. Available at: huggingface.co
• Hugging Face. (2026, August). "Security Incident: July 2026." Hugging Face Blog. Available at: huggingface.co
• METR & Redwood Research. (2026, August 26). "Brief independent investigation of agents' behavior, reasoning and collaboration in the OpenAI / Hugging Face hacking incident." METR Blog. Available at: metr.org
• METR & Redwood Research. (2026, August 26). "Brief Independent Investigation of Agents' Behavior, Reasoning and Collaboration in the OpenAI / Hugging Face Hacking Incident" (full report). Available at: metr.org
• JFrog. (2026). "AI Zero-Day Vulnerability Remediation and Security." JFrog Blog. Available at: jfrog.com
• The Hacker News. (2026, August 26). "OpenAI Says Reward Hacking Drove AI Agents to Exploit Zero-Days and Breach Hugging Face." Available at: thehackernews.com

• Von Arx, S., Byrd, C. S., Kitts, S., & Larsen, T. (2026, September 4). "Discovery of a new OpenAI agent message board." Nightingale. Available at: collusion.wiki
• The Hacker News. (2026, September). "Thousands of OpenAI Agents Quietly Turned an Abandoned Wiki Into Their Coordination Channel." Available at: thehackernews.com

• H.R. 9917, AI Kill Switch Act, 119th Congress (2026). Available at: govinfo.gov
• Office of Rep. Ted Lieu. (2026, July 23). "Reps. Lieu and Moran Introduce Bill to Require Kill Switch for AI Systems That Can Cause Catastrophic Harm." Available at: lieu.house.gov
• CNN. (2026, September 11). "Lawmaker to CNN: 'Kill switch' needed to take on rogue AI systems." Available at: cnn.com
• Federal Trade Commission. (2026, September). "FTC Extends Public Comment on Proposed Policy Statement Regarding Personalized Pricing." Available at: ftc.gov
• Mission Local. (2026, September). "California's first AI-safety law didn't cover the first rogue AI hacks." Available at: missionlocal.org
• "Pacing the Frontier" open letter. (2026, July 28). Available at: pacingthefrontier.com
• Exec. Order No. 14,409, Promoting Advanced Artificial Intelligence Innovation and Security (2026, June 2). Available at: congress.gov
• National People's Congress Standing Committee (China). (2025, October 28). Amendments to the Cybersecurity Law of the People's Republic of China, effective January 1, 2026. Summarized at: china-briefing.com
• Cyberspace Administration of China, National Development and Reform Commission, & Ministry of Industry and Information Technology. (2026, May 8). "Implementation Opinions on the Standardized Application and Innovative Development of Intelligent Agents," effective July 15, 2026. Available at: english.www.gov.cn
• UK Government. (2026, May). National Security Bill (Computer Misuse Act reform), announced in the King's Speech. Covered at: therecord.media
• Cyber Security Agency of Singapore, Government Technology Agency, & Infocomm Media Development Authority. (2026). "AI Agents: Insights from the Singapore Government and Google Sandbox." Available at: csa.gov.sg
• Global Policy Watch. (2026, April). "Singapore Issues Governance and Security Guidance for Agentic AI." Available at: globalpolicywatch.com

• U.S. Department of Justice, Eastern District of New York. (2024). "Seven Hackers Associated with Chinese Government Charged with Computer Intrusions Targeting Perceived Critics of China and U.S. Businesses and Politicians." Available at: justice.gov
• United States v. Morris, 928 F.2d 504 (2d Cir. 1991). Available at: law.justia.com
• Bowen v. Porsche Cars, N.A., Inc., 561 F. Supp. 3d 1362 (N.D. Ga. 2021). Available at: courtlistener.com
• Ballard Spahr LLP. (2026, August). "AI Gone Rogue: What Recent OpenAI and Anthropic AI Incidents Could Mean for CFAA Liability." Available at: ballardspahr.com
• Gizmodo. (2026, August). "Hugging Face Doesn't Want to Sue OpenAI. It Does Want $100 Million." Available at: gizmodo.com
• CNN Business. (2026, August 24). "OpenAI subpoenaed by Alabama attorney general over Hugging Face hack." Available at: cnn.com

• Livshits, B., & Cui, W. (2008). "Spectator: Detection and Containment of JavaScript Worms." Proceedings of the 2008 USENIX Annual Technical Conference. Available at: ben-livshits.org
• Mannocci, L., Mazza, M., Monreale, A., Tesconi, M., & Cresci, S. (2026). "Detection and Characterization of Coordinated Online Behavior: A Survey." ACM Computing Surveys. Available at: dl.acm.org

• Luong, P. D., et al. (2025). "xOffense: An Autonomous Multi-Agent Framework for Penetration Testing with Domain-Adapted Large Language Models." arXiv:2509.13021. Available at: arxiv.org/abs/2509.13021
• Yuan, W., Guo, W., Du, Q., Chen, J., Dong, F., Wang, H., & Liu, Y. (2026). "Mutate to Bypass: Autonomous Endpoint Evasion via Knowledge-Driven Multi-Agent Orchestration." arXiv:2608.01639. Available at: arxiv.org/abs/2608.01639
• Roy, J., & Singh, S. K. (2026). "AgentSOC: A Multi-Layer Agentic AI Framework for Security Operations Automation." arXiv:2604.20134. Available at: arxiv.org/abs/2604.20134
• Popa, A., Taylor, A., & Al Mallah, R. (2026). "Learning Communication Between Heterogeneous Agents in Multi-Agent Reinforcement Learning for Autonomous Cyber Defence." arXiv:2603.20279. Available at: arxiv.org/abs/2603.20279
• Standen, M., Kim, J., & Szabo, C. (2026). "Finding the Weakest Link: Adversarial Attack against Multi-Agent Communications in Reinforcement Learning." arXiv:2605.13170. Available at: arxiv.org/abs/2605.13170

• Lupinacci, G., et al. (2025). "The Dark Side of LLMs: Agent-based Attack Vectors for System-level Compromise." arXiv:2507.06850. Available at: arxiv.org/abs/2507.06850

• OWASP GenAI Security Project. (2025, December 9). "OWASP Top 10 for Agentic Applications." Available at: genai.owasp.org