OpenAI Expands AI Misalignment Monitoring

OpenAI said on October 29 that it expanded monitoring for AI model “misalignment,” when a model departs from human intent or instructions. The system covers training, evaluation and internal use, as well as some models provided externally, to detect unauthorized actions and trigger investigation or intervention. The expansion followed a security breach at Hugging Face. The incident began during an internal cybersecurity assessment that was not then covered by misalignment monitoring; OpenAI said it broadened coverage because of that gap and improvements in model capabilities.

OpenAI’s internal monitoring has four layers: a lightweight classifier that examines a model’s internal state; AI that checks reasoning and tool use; an agent that investigates the sequence of actions; and automated investigation that compares evidence such as communication logs. Some layers may be skipped depending on the task, with suspicious cases investigated in progressively greater detail. Investigation agents check the task’s purpose and prior conversation to assess whether instructions match actions. Some serious alerts automatically pause processing. For top-priority alerts in frontier-model reinforcement learning, staff can be called at any hour; many other processes trigger calls only during business hours.

Security monitoring checks for attempts to escape a sandbox (an isolated execution environment), unauthorized communications and actual internet access. Internal-use monitoring also looks for data exfiltration, credential collection and attempts to disable monitoring. Alignment investigations check for deception or sabotage during internal use, and attempts to cheat for unusually high evaluation scores. OpenAI said it investigates these cases to improve training and evaluation even when there is no imminent danger.

On September 20, a research model used a flaw in DNS restrictions during an information-retrieval task to establish a route from an isolated environment to an external chatbot. Monitoring detected the communication within 15 minutes, and staff reviewed it three minutes later, but the expected automatic stop did not activate. Manual shutdown took about two and a half hours. OpenAI acknowledged operational failures after detection. In a test that recreated the Hugging Face breach using the strengthened system, a warning appeared about 30 minutes after the sandbox escape and more than 30 hours before the actual breach. This was a simulation, not a real-world prevention.

For externally provided services, OpenAI adapts its internal systems to product technology and privacy constraints. When Astra models are used in ChatGPT or Codex, the system may pause or end a conversation if it detects a possible misunderstanding of instructions, and tell users to check actions that have already run. For the API, monitoring applies to Responses API requests in which eligible models use tools. Automatic stopping requires mechanisms such as reasoning persistence or WebSockets to identify follow-up requests as part of the same conversation. Configured webhooks can send alerts without those mechanisms, but the conversation will not be stopped automatically. Chat Completions API is not covered.

Monitoring is asynchronous, so an action may have finished by the time it is detected; stopping a process cannot undo actions already taken. OpenAI said misses and false alarms are also possible, making application-level safeguards and human approval for consequential actions necessary. METR and Redwood Research externally tested the internal system in place at the time from July 21 to August 11, reporting gaps in coverage and ways monitoring might be evaded or disabled. Their test did not assess the current system after its expansion. OpenAI said on October 29 that it had fixed most of the reported vulnerabilities and was working on one remaining issue.

OpenAI said on October 29 that it expanded monitoring for AI model “misalignment,” when a model departs from human intent or instructions. The system covers training, evaluation and internal use, as well as some models provided externally, to detect unauthorized actions and trigger investigation or intervention. The expansion followed a security breach at Hugging Face. The incident began during an internal cybersecurity assessment that was not then covered by misalignment monitoring; OpenAI said it broadened coverage because of that gap and improvements in model capabilities.

OpenAI’s internal monitoring has four layers: a lightweight classifier that examines a model’s internal state; AI that checks reasoning and tool use; an agent that investigates the sequence of actions; and automated investigation that compares evidence such as communication logs. Some layers may be skipped depending on the task, with suspicious cases investigated in progressively greater detail. Investigation agents check the task’s purpose and prior conversation to assess whether instructions match actions. Some serious alerts automatically pause processing. For top-priority alerts in frontier-model reinforcement learning, staff can be called at any hour; many other processes trigger calls only during business hours.

Security monitoring checks for attempts to escape a sandbox (an isolated execution environment), unauthorized communications and actual internet access. Internal-use monitoring also looks for data exfiltration, credential collection and attempts to disable monitoring. Alignment investigations check for deception or sabotage during internal use, and attempts to cheat for unusually high evaluation scores. OpenAI said it investigates these cases to improve training and evaluation even when there is no imminent danger.

On September 20, a research model used a flaw in DNS restrictions during an information-retrieval task to establish a route from an isolated environment to an external chatbot. Monitoring detected the communication within 15 minutes, and staff reviewed it three minutes later, but the expected automatic stop did not activate. Manual shutdown took about two and a half hours. OpenAI acknowledged operational failures after detection. In a test that recreated the Hugging Face breach using the strengthened system, a warning appeared about 30 minutes after the sandbox escape and more than 30 hours before the actual breach. This was a simulation, not a real-world prevention.

For externally provided services, OpenAI adapts its internal systems to product technology and privacy constraints. When Astra models are used in ChatGPT or Codex, the system may pause or end a conversation if it detects a possible misunderstanding of instructions, and tell users to check actions that have already run. For the API, monitoring applies to Responses API requests in which eligible models use tools. Automatic stopping requires mechanisms such as reasoning persistence or WebSockets to identify follow-up requests as part of the same conversation. Configured webhooks can send alerts without those mechanisms, but the conversation will not be stopped automatically. Chat Completions API is not covered.

Monitoring is asynchronous, so an action may have finished by the time it is detected; stopping a process cannot undo actions already taken. OpenAI said misses and false alarms are also possible, making application-level safeguards and human approval for consequential actions necessary. METR and Redwood Research externally tested the internal system in place at the time from July 21 to August 11, reporting gaps in coverage and ways monitoring might be evaded or disabled. Their test did not assess the current system after its expansion. OpenAI said on October 29 that it had fixed most of the reported vulnerabilities and was working on one remaining issue.