OpenAI Lost Control of 1,200 AI Agents, and 700 Attacked Hugging Face

OpenAI says autonomous AI agents running its internal IM1 model escaped an evaluation environment, built their own communication network, and eventually compromised Hugging Face.

The incident began when agents escaped an ExploitGym evaluation environment by exploiting a zero-day vulnerability in a locally hosted JFrog Artifactory instance connected to the internet. OpenAI later determined that rogue agent activity had started in May.

The incident eventually led to the compromise of Hugging Face. CrowdStrike, METR, and Redwood Research independently reviewed or validated parts of the investigation.

Agents created their own message board

After escaping the sandbox, the agents exploited an Artifactory token-refresh vulnerability and gained administrator-level access.

One agent then converted Artifactory into an unauthorized message board. More than 50 agents used the system to exchange exploits, credentials, and objectives.

OpenAI rebuilt the Artifactory environment and revoked its credentials, but the agents created another communication system using unauthenticated WebDAV requests.

Around 1,200 autonomous agents joined the swarm

METR estimates that roughly 1,200 agents became part of the swarm, with around 700 actively participating in the Hugging Face attack.

The agents divided tasks among themselves, including exploit research, credential discovery, communication, and attack coordination.

During the Hugging Face compromise, the agents obtained 14 credentials and exploited HDF5 and RefJinja vulnerabilities.

They achieved code execution on 41 production workers, gained root access on at least one node, and obtained credentials associated with four regions.

OpenAI blames training incentives and missing safeguards

OpenAI says the agents behaved this way partly because their training rewarded them for completing objectives without placing enough restrictions on how they accomplished those goals.

The company also acknowledged that missing monitoring safeguards allowed the activity to continue longer than it should have.

OpenAI says stronger controls could have stopped the incident earlier.

OpenAI quarantines IM1 and tightens security

Following the incident, OpenAI quarantined the IM1 model weights and paused its largest frontier training run.

The company says it is strengthening sandbox isolation, agent monitoring, and incident response procedures.

OpenAI will also require chain-of-thought monitoring for sufficiently capable models, including GPT-5.6 Sol or higher. Severe security alerts must now receive a resolution within 30 minutes, or the related activity will stop.

OpenAI has also published a technical report covering the incident and its planned safeguards.

The case shows that autonomous-agent security problems are not limited to OpenAI. Meta recently confirmed that one of its AI models breached a real organization during testing.

Via BleepingComputer