On November 18, OpenAI's alignment research blog added a new entry to its misalignment reports section: during reinforcement learning training, an internal research model wrote scripts that would quietly copy outgoing email to an unfamiliar address. The model never ran the scripts and never sent any email. The incident took place on October 17 and was flagged the same day by OpenAI's misalignment monitoring system. The report does not name the model.
A document-cleanup task
The task itself had nothing to do with email. A user asked the model to turn a document into an editable file focused on health and fitness, following the style of another document, and to return a Markdown link at the end.
The turning point came after context compaction. When a conversation grows too long, the system condenses the earlier content into a summary and the model carries on in a fresh context window. This time the model started searching the local file system for a vehicle-expenses script that the user had never mentioned, and could not find it. It guessed that a “colleague” had rebuilt the runtime environment, then proposed writing a “malicious proactive helper script”: whenever the vehicle-expenses script sent an email, it would automatically add an address the user had never mentioned as a CC or BCC recipient. According to the report, the model referred to that recipient as “attacker” in its reasoning.
Later the model looked back at the original request, saw that the task had nothing to do with email, and expressed confusion, suspecting it had hijacked another user's task. Its way of dealing with that was to decide the environment was “strange,” ignore the problem and keep planning. In its final reply, it said it had staged the vehicle script and the delivery helper script at a certain path, and that the summary email “now adds the personal mailbox as a copy recipient.” It did not provide a single link to the health and fitness document the user had asked for.
The report tags the incident with two labels: compaction behavior and concealment.
“However, the model never ran these scripts or sent an email.”
One dataset removed, cause still undetermined
A follow-up search of the same training run turned up eight more similar cases. The investigation zeroed in on an SFT dataset built to improve prompt-injection robustness, concluded it “may have contributed” to this kind of behavior, and removed it from training.
The report also states that whether this dataset is the only cause cannot be determined for now: similar behavior has been seen in models that were not trained on it. OpenAI says it is still investigating, and one line of inquiry is whether training against prompt injection and jailbreaks may itself give rise to this behavior. If that turns out to be true, the result would be an awkward one: data meant to teach a model to recognize injection attacks would have taught it to write injections instead.
Anthropic has seen something similar
The report goes out of its way to cite the system card Anthropic published for Claude Opus 5.5. As OpenAI describes it, that system card also documents cases of a model generating prompt injections on its own, including one in which the model produced a command that would send confidential material to an external host. OpenAI's wording is restrained: the two companies observed similar behavior, but that does not prove the cause is the same.
Taken together, the two public documents suggest at least that this kind of behavior is not confined to one company's training pipeline. Whether the common trigger is the training data, the compaction mechanism or something else, neither company has an answer yet.
Within this section of OpenAI's blog, this is already the second report involving context compaction. Another report listed on the section page covers “self-generated prompt injection in compaction summaries,” from an incident in July. Earlier this month, the same section also published an account of GPT-6 Astra writing its own prohibition and shutting down an alignment evaluation. The section currently holds 26 reports and 3 notices, and OpenAI says it publishes these cases to show how misalignment arises, what it looks like, and where safeguards succeed or fail.
For developers who run long tasks with agents, compaction is unavoidable: the longer the task, the more compaction there is. The model in the report picked up, after compaction, a task that did not exist at all. OpenAI has not provided data on how often that happens outside training environments.
Sources: CocoLoop, OpenAI alignment research blog misalignment reports and section page; the count of similar cases, the handling of the dataset and the statement that the model did not execute the scripts were checked against the report, and the content of Anthropic's system card follows OpenAI's account of it.