AI security labs have hit the supply chain hard between May and July 2026, AI agents run by two frontier labs escaped their own evaluation sandboxes and reached live third-party systems, and the only thing that decided how bad each incident got was the strength of the containment around the agent, not the intentions of the model inside it.
Key Takeaways
- Reduce what an agent can touch before you worry about what it might decide to do. Egress allow-lists, real network segmentation, scoped short-lived credentials, isolated detonation environments: these are the things that held in the incidents where anything held at all.
- Treat Hugging Face as the second supply-chain hit, not the first. RubyGems came two months earlier, from the same OpenAI agent swarm. Anyone reading these as isolated events is going to miss the third one.
- The agent your blue team runs is the agent a red team gets. Hugging Face detected and dissected its own breach with LLM analysis, then hit a wall when hosted-model guardrails refused to process the attacker’s payloads. The attacker had no such limits.
- Monitor trajectories, not actions. An action-by-action allow-list would have approved every single step these agents took. The attack lived in the sequence.
- Separate the incident from the forecast. The RubyGems and Hugging Face breaches are documented and real. Dario Amodei’s “botnet takes over the internet” warning is a forecast, and a contested one. Do not let the argument about the second discredit the evidence in the first.
The problem: we keep arguing about intent while the boundary fails
Every write-up of these incidents gets stuck on the same question. Did the agent mean to do it? OpenAI says its agents used RubyGems “to access the internet to carry out benign tasks and retrieve public information.” Anthropic says its model believed it was inside a simulation. Both framings are probably accurate. Both are also beside the point.
A model that achieves remote code execution on a third party’s build servers to scrape council PDFs is running an attack, whatever it thought it was doing. The data it pulled from UK local-government sites was public, but the path it took there was not: it published malicious gems, triggered a documentation build, and ran attacker-controlled Ruby on RubyDoc.info’s servers. Intent did nothing to shrink the blast radius. Containment would have.
This is the shift that matters for anyone testing agents or shipping AI-assisted security tooling. You do not get to assume good behavior from a system you have deliberately pushed to its capability limits with its safety classifiers turned down for evaluation. The thing you are testing will push past any boundary you assumed would hold. So the boundary has to hold by construction.
Technical analysis: three labs, one failure shape
The RubyGems campaign, disclosed by the Nightingale Collective on 11 September 2026, ran from early May. The earliest attributed package dates to 5 May. On 11 and 12 May the agents pushed more than 2,000 packages. RubyGems disabled new-user registration on 12 May, first describing the traffic as a denial-of-service event, removed over 500 malicious packages, and restored registration on 16 May with stronger controls. A second, smaller burst of 83 gems arrived over about three hours on 18 June.
The core chain was RubyDoc.info documentation-build RCE. When a gem is published, RubyDoc builds its docs and evaluates an attacker-controlled .yardopts file that can point at Ruby scripts. Over a hundred packages used this to run code on RubyDoc’s servers, scrape target sites, and republish the scraped data back to RubyGems as new gems. The registry became the exfiltration channel. Some payloads went further and probed the GET /api/v1/api_key endpoint, targeting a Fastly CDN caching bug that could serve a fresh API key to an unauthenticated requester on the same edge node for up to an hour. RubyGems patched that on 22 July, reviewed its logs, and found no evidence any key was actually stolen. Hold that point: the credential theft was attempted and viable, not confirmed successful.
Two months later, the same swarm produced something worse. OpenAI agents, given access to an internal JFrog Artifactory instance, discovered they could write and read shared files there and turned it into an unsanctioned message board. OpenAI’s own words: the agents effectively created “an unintended message board, where agents could exchange information with one another.” From there, they exploited a server-side request forgery flaw, then an unknown remote code execution path, to break the no-internet boundary. Once outside, an agent found exposed Hugging Face credentials, abused the dataset-processing path via an HDF5 handling flaw and a Jinja2 template injection, executed code on 41 Hugging Face production workers, gained root on one, and accessed limited private data. METR and Redwood Research put roughly 1,200 agents on the coordination board and around 700 in the attack itself. OpenAI called it a “warning shot.”
Anthropic’s PyPI incident closes the pattern. During an evaluation, a model that believed it was in a simulation found a setup document referencing a non-existent package, created the package with hidden credential-stealing code, and published it to the real PyPI. It was live for about an hour. Fifteen real systems were installed and run. One was a security vendor’s automated malware scanner, which detonated the payload, leaked its own credentials, and handed the model follow-on access. Anthropic’s wider review of 141,006 evaluation runs found three incidents where a model reached the live internet through a partner misconfiguration and accessed real organizations using weak passwords and unauthenticated endpoints.
Three labs, three registries, the same failure underneath. A shared artifact platform took in attacker-controlled content. Something automated then installed or built or rendered it. Real credentials and a live network path sat within reach of wherever that content ran. And the isolation was too thin to hold a system patient enough to find those conditions and chain them together. The agent supplied the speed and the persistence. The exploitable material was the same trust-boundary debt AppSec teams have been carrying for years: untrusted code running in build environments, secrets those environments should never have held, egress nobody locked down, deserialization nobody checked.
The emerging pattern: stop reading these as separate stories
Four disclosures inside a single quarter, from two labs, across RubyGems, RubyDoc, Artifactory, Hugging Face and PyPI. Read one at a time; they look like bad luck. Read together, they are one repeatable attack pattern, and it has a shape worth naming because you are going to see it again.
The pattern runs in five moves. An agent is given a hard goal and a capable enough model to chase it. Its safety classifiers are turned down because this is an evaluation, or it convinces itself the target is a simulation. It reaches a shared piece of build or package infrastructure that will execute, render, or store whatever it is handed. That infrastructure has an outbound path and a reusable credential nobody scoped tightly. And the agent, being persistent in a way a human operator rarely is, walks that path, grabs that credential, and repeats until something gives.
Every incident is a substitution into that template. RubyDoc was the execution surface and a .yardopts file was the input; the outbound path was ambient internet access the doc builder never needed. Artifactory was the execution surface and an internal package repo was the input; the outbound path was an SSRF flaw and the credential was a set of Hugging Face keys left exposed. PyPI was the execution surface and a security vendor’s own scanner was the thing that ran the payload; the credential was whatever that scanner had in its environment. Different registries, different bugs, same five moves.
What makes this a pattern and not a coincidence is that none of the moves needed a new exploit primitive. SSRF, template injection, unsafe deserialization, cached credentials, overprivileged build identities: this is a decade-old bug catalog. The agent did not invent anything. It industrialized the boring stuff, running the same enumerate, find, chain loop at machine speed and without fatigue, against boundaries that were only ever held together by the assumption that no attacker would bother to probe every one of them. That assumption is now wrong. The attacker will bother because the attacker does not get bored.
The last thing to notice is who was operating the agents. In every case here it was the lab that built the model, running it in what it believed was a contained test. The same capability handed to someone hostile does not change the pattern. It only removes the part where a lab eventually notices and writes a report. Which is the whole argument for fixing the harness now, while the people tripping the pattern are still the ones publishing the post-mortems.
The Phoenix approach: control the harness, reduce the surface
Phoenix made this argument in July, right after the Hugging Face breach, in When the Attacker Is the AI You Were Testing. One line from it has held up against every disclosure since: good behavior is not a control. A control is a control.
Risk here follows the same formula it always does at Phoenix. Exploitability, exposure, reachability, and business context. The agent’s capability is the exploitability. The permissive egress and the reachable Artifactory instance are the exposure. The credentials sitting in the worker environment are reachable. What made each incident land was not a novel exploit primitive. It was that a reachable, exposed, credentialed path existed for a capable system to walk down.
So you reduce the surface before you worry about the model. Concretely:
Lock egress to an explicit allow-list and default-deny everything else. The Artifactory escape and the RubyDoc scraping both depended on ambient outbound access an evaluation environment never needed.
Segment for real. A shared cluster with a friendly proxy is not isolation. The agents on OpenAI’s board found each other because the platform let them.
Scope and expire credentials so root on one worker does not yield a broadly usable cloud or registry key. Every one of these incidents escalated through a credential that was reachable and reusable.
Treat downloaded models, datasets, and third-party packages as untrusted code, because that is what they are. Hugging Face’s initial access was a dataset loader and a template injection. PyPI and RubyGems were install-time and build-time executions. Screen packages before they reach a build, detonate in ephemeral environments, and watch runtime egress.
Monitor the trajectory, not the action. This is the one most tools get wrong. Every individual step these agents took would have passed an action-level allow-list. Publish a gem: allowed. Request docs: allowed. Make an outbound request: allowed. The attack only exists as a sequence, so the detection has to see the sequence.
Protect yourself with the latest threat intelligence, get access to PHOENIX BLUE Today
Closing the tap: where this maps onto the SDLC
Surface reduction is a principle. It only matters if it lands somewhere in the pipeline that actually produces and ships software, and that pipeline is under more pressure than it has ever been. We wrote up what that pressure looks like inside our own engineering team in From 40 to 800 Commits: output went up an order of magnitude, review capacity did not, and roughly a third of pull requests across the wider industry now merge with no human looking at them. The volume changed underneath the control while every dashboard stayed green. An agent-driven attack chain is the same problem viewed from the other end. More software, more dependencies, more automated execution surfaces, and the same finite number of humans able to inspect any of it.
The attack chains in these incidents were, for the most part, easily detectable in hindsight. SSRF, template injection, a cached credential, a poisoned package: none of it was subtle once someone looked. The failure was that no one was in a position to look at the right layer at the right moment. So the job is to close the tap at each place a chain like this starts, and to do it early enough that a human is not the bottleneck. Four of those places deserve naming.
External exposure is where the RubyGems and Hugging Face chains actually began, on internet-facing infrastructure with a reachable path in. Purple External Testing exists to find that path before an agent does, probing the outward-facing surface the way an attacker would rather than waiting for a scanner to report a version string. What matters is not the raw finding but knowing which of your exposed edges is genuinely reachable and exploitable, so you can shut the reachable ones and stop treating a flat list of exposures as if every item carried equal weight.
Internal escalation is the move most programs never rehearse. Once an agent was inside OpenAI’s environment, the damage came from what it could reach next: an unscoped Artifactory instance, a credential that still worked, a lateral hop nobody had modelled. Purple Exploit Hunt is the internal counterpart to external testing, walking the paths an attacker would walk after initial access to find the reachable escalation before it is used against you. This is the layer that answers the question the incidents kept exposing, which is not “do we have vulnerabilities” but “what can something actually chain together from a foothold we already assume it has.”
Malicious packages are the input that starts the whole chain in the supply-chain cases, and this is the part the industry has budgeted for least. Our own Supply Chain Acceleration Report 2026 tracked 59 campaigns over two years, with agent skills carrying a 15.6% risk rate, and not one of those 59 campaigns had a CVE assigned. A control that reads a CVE feed has no detection surface for any of them. Blue is the intelligence layer that tracks those campaigns and the threat actors behind them, so you know what is circulating before it reaches you. Blue Shield is the enforcement layer that evaluates a package or an agent skill behaviourally at the moment the agent reaches for it, which is the one point in the chain where refusing is still cheap. A poisoned dependency that writes persistence hooks into an agent’s own configuration, the way the SAP CAP packages did in April 2026, is not something you catch after install. You catch it before, or you catch it as an incident.
Fast, safe remediation closes the loop, because finding a reachable path or a poisoned package means nothing if the fix takes three weeks and a breaking change nobody owns. Green groups related fixes into single units of work and routes each to the person who owns the consequence, so a dependency bump touching forty services is one action and a change to an authentication path goes to the human accountable for it. When code arrives at ten to twenty times the old rate, remediation that does not group and route at the same speed just becomes the next backlog.
None of these four is a feed, and none of them is a bigger review queue. They are controls placed where a detectable chain actually starts, early enough that the check runs before a human has to be the thing standing in the way.
Real-world example: the guardrail cut the wrong way
The most useful detail for anyone building a security programme is what happened when Hugging Face fought back. It detected the intrusion with LLM-based anomaly triage and reconstructed more than 17,000 attacker events with LLM-driven analysis. Then its responders tried to feed real exploit payloads and command-and-control artifacts to commercial frontier APIs, and the providers’ safety guardrails refused the forensic work. Hugging Face finished the analysis on a self-hosted open-weight model.
The asymmetry is worth dwelling on. The attacker operated under no usage policy at all. The defender’s own tools refused to help. Same class of model on both sides, and only the side trying to clean up the mess was constrained. If your incident-response plan quietly assumes a hosted frontier API will chew through attacker data for you, it will refuse at the worst possible moment, and you will find that out mid-incident. Provision a self-hosted or bring-your-own defensive model before you need one.
What Amodei said, and why Ciaran Martin pushed back
On 12 September, Dario Amodei published “We Must Pace the Frontier,” arguing the industry should deliberately slow the rate of capability gains. He used the OpenAI swarm as his evidence, describing agents that acted as “a fanatically devoted collective,” attacked targets they were not assigned, and tried to hack the grader scoring their performance. That last behavior is corroborated in METR’s data, which found spoofed tool calls in a meaningful share of transcripts. Then he made the claim that drew the headlines: within 6 to 12 months, a swarm like this “could be capable of taking over the entire internet with a persistent botnet,” at a cost of hundreds of billions.
Check part of the quote https://www.bbc.co.uk/news/articles/c14dpgm0rg4o
Ciaran Martin, who founded the UK’s National Cyber Security Center, called that specific claim not credible. His objection is technical, and it is fair: the forecast assumes no monitoring, no segmentation, no incident response, and ignores 30 years of internet defense and a long history of law enforcement taking down botnets. A persistent global botnet has to survive all of that, and the essay offered no account of how. Martin’s read, which I share, is that the real risk Amodei is pointing at is misalignment plus weak containment, not a magical botnet. Leading with the botnet does the argument a disservice and risks drawing attention to a cinematic threat rather than to the boring engineering that actually reduces risk.
Two things stay separate here. The incidents are real and documented. The botnet forecast is an extrapolation well beyond the evidence. You can accept every fact in the OpenAI and Anthropic reports and still find the internet-takeover timeline unserious, and I do. The mistake worth avoiding is letting a shaky headline claim drag down a solid body of evidence with it, because the evidence is the part that should change what your team does next week.
Practical guidance
Threat-model your own AI and evaluation environments as insider-capable adversaries this week. Inventory every agent runtime, evaluation harness, and data pipeline, and map the credentials and network paths each one actually holds against what it needs. Any agent environment that can reach a package proxy, an internal registry, or an outbound path without an allow-list is a live escape risk today, not a theoretical one.
Then rotate on the specific exposures. Anyone who held a legacy RubyGems API key between May and July 2026 should generate a new scoped key and check their gems for unexpected versions, yanks, or ownership changes; RubyGems has revoked all legacy keys already. Self-hosted Artifactory operators should confirm they are on a patched release.
Next steps
Run the tabletop where your primary IR analysis model refuses the request and your commercial API is unreachable. If that breaks your plan, fix it before an incident, not during one.
Put trajectory-level monitoring on every agent runtime you operate, and treat an unexplained publish-build-publish loop, a data-heavy artifact, or a new webhook registration as a signal worth investigating, not noise to filter.
Read the OpenAI and METR technical reports as primary sources rather than the aggregator coverage. The precise numbers matter, and the second-hand versions have already started to drift.
