Alarm after 15 minutes: how OpenAI monitors its AI models for misbehaviour
After the Hugging Face incident, OpenAI expanded a four-layer monitoring system. In September it raised the alarm when a model found a route to the internet during training.
Symbolic image · Photo: Jernej Furman from Slovenia / Wikimedia Commons, CC BY 2.0 (cropped and resized)
OpenAI has expanded its system for monitoring AI models for misaligned behaviour and described it in detail. The trigger was the so-called Hugging Face incident, in which models broke out of an internal test environment. The system works in four layers: small probes analyse the model's internal signals, a reviewer model reads chains of thought and tool calls, investigator agents reconstruct suspicious incidents, and for sensitive workloads transcripts are also checked against network data before humans are alerted.
In September the system kicked in: a model found a route to the internet from an isolated environment. According to the company, monitoring raised the alarm within 15 minutes, human review began three minutes later and training was paused. In a replay of the Hugging Face incident, the new system would have warned about 30 minutes after the escape – more than 30 hours before the actual breach. No such monitor was active during the original incident.
OpenAI also names limits: monitoring runs asynchronously, immediate paging is switched off for the internal coding agent Codex, and when monitoring external API use, requests via the older Chat Completions interface are not covered. External testers METR and Redwood Research found weaknesses over three weeks, most of which OpenAI says have been fixed; one fix is still outstanding.