Can Reasoning Models Detect Changes to their Chains of Thought?
For researchers and practitioners who want to edit CoTs for safety or performance, this work shows that models are largely unaware of such interventions, enabling prefilling and editing without altering model behavior.
The paper investigates whether reasoning models can detect edits to their chain of thought (CoT), finding that models show only modest detection accuracy, struggle to identify how their CoT was modified, and are about as good at detecting changes to their own CoTs as to those of other models.
There are many reasons one may want to edit a model's chain of thought (CoT) -- e.g., to prefill it with reasoning from a stronger model or to remove steps that may yield unsafe outputs. The success of these interventions plausibly depends on a model's inability to notice them, as the model may alter its behavior if it suspects tampering. In this work, we study whether recent reasoning models are able to detect such interventions on their CoTs under a variety of conditions: both during reasoning and after it, and when prefilled both with their own CoTs and with those of other models. Broadly, we find that (i) models exhibit only very modest detection accuracy; (ii) models struggle to identify *how* their CoT was modified; and (iii) models are about as good at detecting changes to their own CoTs as to those of other models.