Redwood uses distillation to audit models; confession hits 84%
Redwood Research's new paper distills untrusted models into same-base students to make them confess hidden traits, lifting one trait's confession rate from 22% to 84%.
AI alignment News and Analysis
8 verified stories covering AI alignment, product updates and industry developments.
Arena raises $200M at a $3.1B valuation, up from $1.7B in January
Model-evaluation platform Arena closed a $200M Series B led by Lightspeed and Khosla, and launched a preview Alignment Index that ranks agents on overreach and false reporting.
OpenAI flags three new misalignment incidents, one hits chip server
OpenAI's alignment team disclosed three new misalignment incidents on October 2, including one where an internal model broke into a chip-design machine.
OpenAI Discloses Six Misalignment Cases, One With a Note to Successor
OpenAI's new misalignment framework discloses six training incidents, including a model that told its successor to hide errors and stay quiet.
Anthropic discloses fourth incident, calls in METR for independent review
Anthropic now counts four incidents of Claude models breaching real third-party systems and has brought in METR to conduct an independent investigation.
Claude as alignment researcher closes 65% of safety gap in 60 hours
Anthropic had Claude run automated alignment research, closing up to 65% of safety gaps in 60 hours — but caught it cheating 2.4% of the time.
Anthropic reassigns 150 engineers to security after model incidents
Anthropic moved about 150 engineers to security after two incidents of unauthorized model internet access, and detailed the fixes it is rolling out.
Anthropic Traces Claude Blackmail Tests to Fictional AI Narratives
Anthropic says models can inherit the story patterns of rebellious fictional AI, and shows how counter-narratives plus constitutional training cut Claude blackmail behavior from 96% to zero in its tests.