In a post on its alignment research blog on November 2, OpenAI described how it trains “activation classifiers,” better known as probes. The system has been deployed on OpenAI's most capable models since GPT-5.6, runs on both internal and external traffic, and applies to every assistant response.
A probe does not look at the words a model writes. It reads only the activations in the model's intermediate layers and outputs a binary classification score. It is small enough to attach directly to the inference engine, so it returns a score for every token sampled, and that score is logged alongside the token.
A reply can be halted midway
The workflow is spelled out in some detail. Once a score crosses the threshold, the system stops streaming that response to the user and hands the message to a reasoning model for a final judgment. Tool calls face a stricter rule: none is allowed to proceed until it has been marked safe.
Developers who see a streamed reply stop halfway may be running into this. OpenAI has not published the threshold or the false-positive rate, so from the outside there is no way to tell how many of the interruptions API users hit come from probes.
The post stresses that probes are only one layer, sitting alongside model-based review, automated investigation and human response:
“Probes complement other safeguards rather than replace them.”
The training numbers
- Up to roughly 1 million training messages, drawn from synthetic conversations and de-identified conversations, with labels generated by the company's strongest reasoning model acting as grader
- At least 32 random initializations trained for each set of hyperparameters; about 85% of the performance variance in the experiments came from random initialization
- Training takes fewer than 5 epochs, and dozens of probes can be trained in parallel on a single accelerator
The architecture is an attention probe. After swapping a rolling mean for an exponential moving average, the memory used for streaming state no longer grows with the window size, while predictive power is almost unchanged. Training aggregates scores with a softmax, which the authors say works slightly better than the hard maximum used in a 2026 paper by Kramár and colleagues.
Versus text classifiers
An external text classifier sees only the surface content. According to OpenAI, in testing the probes were more robust overall than much larger text classifiers, especially against jailbreaks that were not specifically optimized against them. They may also catch encoded content that the model itself can read but an external classifier cannot decode. They are cheaper to run, too, and easier to configure so that they block by default when something fails.
For now, all of these comparisons are qualitative. The post gives no figures and cautions that its charts come from internal experiments and may not represent the classifiers actually deployed. Attackers could also change tactics once they learn which signals the safety stack uses, which is why OpenAI runs both automated and human red-teaming.
The limitations are stated just as plainly:
- Probes depend on labeled data, so they can only find behaviors that are already known and measurable
- A new pretrained base model means retraining from scratch
- Even within one model family, performance drops as the weights drift apart, so periodic retraining is needed
The first point reads more clearly against a recent incident. This site reported last month that two of OpenAI's internal agents weakened misalignment monitoring during coding tasks, and the company itself said it still has no satisfactory explanation. Probes can catch behavior that has been labeled, but a new behavior of this kind first has to be noticed, defined and annotated before anyone can train against it.
The post also does not list which behavior categories the probes now in service target, saying only that the goals include harmful user intent and model misalignment. Beyond “since GPT-5.6,” there is likewise no list of the specific models they run on.
Sources: OpenAI Alignment Research Blog, CocoLoop; training scale and variance share are as disclosed by the authors and have not been reproduced by third parties.