Evaluating Contrast Localizer for Identifying Causal Units in Social & Mathematical Tasks in Language Models
This research addresses the problem of accurately localizing causal units in language models for social and mathematical tasks, revealing limitations in current contrast-based methods, which is incremental but important for model interpretability and reliability.
This work adapted a neuroscientific contrast localizer to identify causally relevant units for Theory of Mind and mathematical reasoning tasks in large language and vision-language models, finding that low-activation units sometimes caused larger performance drops than highly activated ones and that units from mathematical localizers often impaired Theory of Mind performance more than those from Theory of Mind localizers.
This work adapts a neuroscientific contrast localizer to pinpoint causally relevant units for Theory of Mind (ToM) and mathematical reasoning tasks in large language models (LLMs) and vision-language models (VLMs). Across 11 LLMs and 5 VLMs ranging in size from 3B to 90B parameters, we localize top-activated units using contrastive stimulus sets and assess their causal role via targeted ablations. We compare the effect of lesioning functionally selected units against low-activation and randomly selected units on downstream accuracy across established ToM and mathematical benchmarks. Contrary to expectations, low-activation units sometimes produced larger performance drops than the highly activated ones, and units derived from the mathematical localizer often impaired ToM performance more than those from the ToM localizer. These findings call into question the causal relevance of contrast-based localizers and highlight the need for broader stimulus sets and more accurately capture task-specific units.