Localizing Anchoring Pathways in Language Models
Provides a mechanistic understanding of how anchoring biases are encoded in language models, relevant for AI safety and interpretability researchers.
The paper studies anchoring effects in language models, where irrelevant numbers in prompts shift numerical reasoning. Using circuit localization on 7B-8B models, they find edge-level methods better track anchoring signals than node-level methods, with strong transfer within models but less reliable transfer across base and instruction-tuned variants.
Irrelevant numbers in a prompt can shift language model judgments, producing anchoring effects in numerical reasoning. We study where this anchor-sensitive signal is carried inside language models using a controlled multiple-choice setup with shared answer options. We define a logit-difference metric comparing the correct answer option with the answer option corresponding to the anchor, and validate that it tracks behavioral anchoring. Using attribution-based circuit localization on 7B--8B Qwen and Llama base and instruction-tuned models, we find that edge-level methods recover this signal more faithfully than node-level methods. Low- and high-anchor circuits transfer strongly within a model, suggesting shared pathway structure across anchor direction. However, sparse transfer across base and instruction-tuned variants is less reliable, indicating that post-training changes which pathways matter most. Overall, our results provide a mechanistic account of how anchoring-related decision signals are carried inside language models.