Connect with us

Hi, what are you looking for?

SecurityWeekSecurityWeek

Artificial Intelligence

OpenAI Overhauls Model Security With Sandboxing, 30-Minute Alerts, and Training Pauses

The action taken by OpenAI comes in light of the Hugging Face incident and the discovery of the Astra model’s advanced capabilities. 

OpenAI

OpenAI has detailed new containment and continuous monitoring protocols for its AI research, introducing stricter isolation and a token-inspection system to manage models with advanced cybersecurity capabilities.

These security measures follow internal evaluations indicating that an upcoming model, Astra, may meet the ‘critical’ cybersecurity capability threshold under the organization’s Preparedness Framework. 

That finding, alongside a recent security incident involving Hugging Face, prompted operational delays, including a two-week pause in reinforcement learning training for deployment-bound models and an ongoing hold on its largest planned frontier training run.

Following these developments, OpenAI says it’s restructuring its research infrastructure. Workloads that execute model-generated or untrusted code must now operate within stronger sandboxes. In addition, network boundaries have been reconfigured so that a single workload compromise cannot independently grant unauthorized access to the internet or internal networks.

A central component of the security updates is a multistage monitoring framework. The system uses activation classifiers to inspect a model’s internal activity at every sampled token. 

Anomalies are escalated to automated investigators tasked with analyzing the model’s reasoning sequences and tool usage for data theft, unauthorized access, destructive behavior, or attempts to bypass safeguards.

Advertisement. Scroll to continue reading.

This continuous oversight introduces a strict operational SLA and a substantial compute tax. High-priority alerts generated by the monitoring system immediately page internal security and research teams. If responders cannot conclusively prove the alert is a false positive within 30 minutes, they are required to pause the activity. The company estimates this monitoring layer consumes roughly 20 percent of the monitored inference compute.

The updated monitoring standards are now mandatory for all reinforcement learning training and evaluation involving tools for models at the Sol capability tier or higher. 

The company noted that its existing Preparedness Framework must evolve to address these capability jumps, requiring core alignment techniques to be applied across more stages of the training process. 

Moving forward, OpenAI anticipates that AI models will need to drive the majority of security operations, including defending against other models, to ensure that protections scale effectively alongside future system capabilities.

OpenAI is not the only AI giant whose models hacked real-world systems during cybersecurity evaluations. Anthropic and Meta reported similar incidents, all tied to testing conducted by the AI security firm Irregular, which has begun releasing detailed reports explaining why the incidents occurred. 

Related: OpenAI Fixes ChatGPT Agent Flaw That Could Let Attackers Forge an AI Insider

Related: Industry Reactions to OpenAI Models Hacking Hugging Face: Feedback Friday

Related: Conflicting Test Goals Pushed Claude Agents to Deploy Self-Replicating Malware

Written By

Eduard Kovacs (@EduardKovacs) is senior managing editor at SecurityWeek. He worked as a high school IT teacher before starting a career in journalism in 2011. Eduard holds a bachelor’s degree in industrial informatics and a master’s degree in computer techniques applied in electrical engineering.

Daily Briefing Newsletter

Subscribe to the SecurityWeek Email Briefing for the latest cybersecurity threats, trends, and expert insights.

Trending

Daily Briefing Newsletter

Subscribe to the SecurityWeek Email Briefing to stay informed on the latest threats, trends, and technology, along with insightful columns from industry experts.

Learn how to address potential risks and not restrict AI adoption in your organization. See what a centralized AI gateway is and how it works in practice.

Register

Join as we decipher the world of zero trust and share war stories on securing an organization by eliminating implicit trust and continuously validating every stage of a digital interaction.

Register

People on the Move

Chip Wentz has been appointed as SVP & CISO at Keurig Dr Pepper Inc.

Lumen Technologies has named Kim Keever as CSO.

Quantum Secure Encryption Corp. has appointed Joseph Hall as CIO.

More People On The Move

Expert Insights

Daily Briefing Newsletter

Subscribe to the SecurityWeek Email Briefing to stay informed on the latest cybersecurity news, threats, and expert insights. Unsubscribe at any time.