Virtual Event Today: CodeSecCon - Learn to Secure Your Software > Join Event
Connect with us

Hi, what are you looking for?

SecurityWeekSecurityWeek

Artificial Intelligence

OpenAI Overhauls Model Security With Sandboxing, 30-Minute Alerts, and Training Pauses

The action taken by OpenAI comes in light of the Hugging Face incident and the discovery of the Astra model’s advanced capabilities. 

OpenAI

OpenAI has detailed new containment and continuous monitoring protocols for its AI research, introducing stricter isolation and a token-inspection system to manage models with advanced cybersecurity capabilities.

These security measures follow internal evaluations indicating that an upcoming model, Astra, may meet the ‘critical’ cybersecurity capability threshold under the organization’s Preparedness Framework. 

That finding, alongside a recent security incident involving Hugging Face, prompted operational delays, including a two-week pause in reinforcement learning training for deployment-bound models and an ongoing hold on its largest planned frontier training run.

Following these developments, OpenAI says it’s restructuring its research infrastructure. Workloads that execute model-generated or untrusted code must now operate within stronger sandboxes. In addition, network boundaries have been reconfigured so that a single workload compromise cannot independently grant unauthorized access to the internet or internal networks.

A central component of the security updates is a multistage monitoring framework. The system uses activation classifiers to inspect a model’s internal activity at every sampled token. 

Anomalies are escalated to automated investigators tasked with analyzing the model’s reasoning sequences and tool usage for data theft, unauthorized access, destructive behavior, or attempts to bypass safeguards.

Advertisement. Scroll to continue reading.

This continuous oversight introduces a strict operational SLA and a substantial compute tax. High-priority alerts generated by the monitoring system immediately page internal security and research teams. If responders cannot conclusively prove the alert is a false positive within 30 minutes, they are required to pause the activity. The company estimates this monitoring layer consumes roughly 20 percent of the monitored inference compute.

The updated monitoring standards are now mandatory for all reinforcement learning training and evaluation involving tools for models at the Sol capability tier or higher. 

The company noted that its existing Preparedness Framework must evolve to address these capability jumps, requiring core alignment techniques to be applied across more stages of the training process. 

Moving forward, OpenAI anticipates that AI models will need to drive the majority of security operations, including defending against other models, to ensure that protections scale effectively alongside future system capabilities.

OpenAI is not the only AI giant whose models hacked real-world systems during cybersecurity evaluations. Anthropic and Meta reported similar incidents, all tied to testing conducted by the AI security firm Irregular, which has begun releasing detailed reports explaining why the incidents occurred. 

Related: OpenAI Fixes ChatGPT Agent Flaw That Could Let Attackers Forge an AI Insider

Related: Industry Reactions to OpenAI Models Hacking Hugging Face: Feedback Friday

Related: Conflicting Test Goals Pushed Claude Agents to Deploy Self-Replicating Malware

Written By

Eduard Kovacs (@EduardKovacs) is senior managing editor at SecurityWeek. He worked as a high school IT teacher before starting a career in journalism in 2011. Eduard holds a bachelor’s degree in industrial informatics and a master’s degree in computer techniques applied in electrical engineering.

Daily Briefing Newsletter

Subscribe to the SecurityWeek Email Briefing for the latest cybersecurity threats, trends, and expert insights.

Trending

Daily Briefing Newsletter

Subscribe to the SecurityWeek Email Briefing to stay informed on the latest threats, trends, and technology, along with insightful columns from industry experts.

Join this live webinar as we explore if detection-first security operations can keep pace with AI, or if it’s time to rethink prevention as the strongest default.

Register

CodeSecCon bridges the gap between dev and security. Discover best practices for secure coding, innovative risk-reduction tools, and safe AI integration to cultivate a true DevSecOps culture. Safely secure your apps!

Register

People on the Move

Dali Rajic is joining OpenAI as Chief Revenue Officer.

Erika Dean has been appointed Chief Information Security Officer at Tricentis.

C1 has named Jeff St. Clair Chief Revenue Officer.

More People On The Move

Expert Insights

Daily Briefing Newsletter

Subscribe to the SecurityWeek Email Briefing to stay informed on the latest cybersecurity news, threats, and expert insights. Unsubscribe at any time.