OpenAI used AI agents to hack its own defenses before resuming model training
The company spent hundreds of billions of tokens testing whether teams of AI agents could escape sandboxes, reach the live internet or communicate across isolated environments. The exercise uncovered software vulnerabilities and configuration errors that OpenAI says it has addressed.
OpenAI will train its most advanced AI models without access to the live internet, citing the growing security risks posed by increasingly capable systems that can exploit software vulnerabilities and bypass restrictions designed to contain them. Instead, the company is using an “offline world” of saved and indexed websites that allows models to practice browsing and searching without affecting external systems.
The policy follows a series of incidents that have raised concerns about the ability of advanced AI models and agents to escape their testing environments or act beyond their intended boundaries. In July, an incident involving Hugging Face highlighted the ability of models with sophisticated cyber capabilities to bypass containment. OpenAI says its own research has also revealed ways that models can exploit software flaws, misconfigurations and indirect communication channels to cross security boundaries.
About two weeks ago, Anthropic announced that it had begun restricting its most advanced models’ access to the live internet. OpenAI is now taking a similar step, saying the decision reflects the current limits of research into AI alignment, monitoring and internet security.
“Based on the current state of alignment research, monitoring systems, and the security of internet infrastructure, we believe the responsible approach is to conduct frontier model training with no internet access,” OpenAI said in a statement. The company said it had implemented the policy and would maintain it until improvements in its safeguards justified broader access.
A key factor was an incident on September 20, when an internal OpenAI research model exploited a misconfiguration in a DNS resolver to gain limited access to the live internet during a training run. OpenAI said it contained the incident but found weaknesses that required further work.
The company paused training and evaluation of its most capable models to strengthen its infrastructure, improve monitoring and enhance systems designed to automatically halt runs when potentially dangerous activity is detected. It resumed reinforcement-learning training and internal inference on October 1, 11 days after the pause began.
OpenAI says the incident also underscored a broader problem: as models become more capable, they become better at finding weaknesses in the systems intended to constrain them. A sufficiently sophisticated agent might exploit a software flaw to escape a sandbox, use an internal service to reach the internet or find unauthorized ways to communicate with agents in other isolated environments.
To test its defenses, OpenAI launched a large-scale red-teaming campaign in which teams of AI agents were tasked with finding ways to access the live internet, cross internal security boundaries and communicate with agents from which they were supposed to be isolated. The company spent hundreds of billions of tokens on these experiments, using some of its most capable models and giving them infrastructure source code, configuration details and documentation about potential vulnerabilities.
The experiments uncovered previously unknown software vulnerabilities and misconfigurations. OpenAI said its agents found ways to escape sandboxes, move between systems, reach the live internet and communicate across isolation boundaries. In one experiment, agents in separate virtual machines exchanged synthetic secrets by exploiting differences in processor-cache timing, without opening a network connection or using shared writable files.
The company said it had addressed the weaknesses identified in its infrastructure and worked with the maintainers of affected open-source software to disclose and fix vulnerabilities. It also strengthened network restrictions and monitoring systems. OpenAI said its tests did not uncover a route to the internet that agents could trivially exploit while concealing their activity from monitoring systems, but it expects future models to discover vulnerabilities that remain unknown.
The company’s strategy relies on what it calls “defense in depth,” combining multiple overlapping safeguards rather than relying on a single barrier. These include restricted execution environments, lightweight virtual machines that isolate agents from their host computers, monitoring systems designed to detect unauthorized activity, and network and cloud controls intended to contain a breach even if earlier defenses fail.
The challenge is that models still need to learn how to use the web. Browsing, searching for information and interacting with web applications are important capabilities for AI systems designed to carry out tasks on users’ behalf. Cutting off all web-based learning would limit their ability to develop those skills.
To address that problem, OpenAI has built an offline environment containing a vast collection of cached and indexed web content. Models can search and interact with this material, but they cannot use it to affect live websites or other external systems. OpenAI says all of its frontier reinforcement-learning training runs now use this offline mode.
The company has made continuous red-teaming with its most capable models a core part of its safety assessments for future training and evaluation runs. It plans to keep using AI agents to probe its own infrastructure, identify vulnerabilities and test whether monitoring and containment mechanisms remain effective as models improve.
