On the Failure of Topic-Matched Contrast Baselines in Multi-Directional Refusal Abliteration
This work addresses a methodological issue in refusal removal for instruction-tuned language models, but it is incremental as it focuses on refining contrast baseline design.
The paper tackled the problem of removing refusal behavior from language models by investigating whether topic-matched contrast baselines improve refusal direction extraction, and found that topic-matched contrast failed to produce functional refusal directions at any tested weight or layer, while unmatched contrast achieved complete refusal elimination on six layers.
Inasmuch as the removal of refusal behavior from instruction-tuned language models by directional abliteration requires the extraction of refusal-mediating directions from the residual stream activation space, and inasmuch as the construction of the contrast baseline against which harmful prompt activations are compared has been treated in the existing literature as an implementation detail rather than a methodological concern, the present work investigates whether a topically matched contrast baseline yields superior refusal directions. The investigation is carried out on the Qwen~3.5 2B model using per-category matched prompt pairs, per-class Self-Organizing Map extraction, and Singular Value Decomposition orthogonalization. It was found that topic-matched contrast produces no functional refusal directions at any tested weight level on any tested layer, while unmatched contrast on the same model, same extraction code, and same evaluation protocol achieves complete refusal elimination on six layers. The geometric analysis of the failure establishes that topic-matched subtraction cancels the dominant activation component shared between harmful and harmless prompts of the same subject, reducing the extracted direction magnitude below the threshold at which weight-matrix projection perturbs the residual stream. The implications for the design of contrast baselines in abliteration research are discussed.