Linear Probes Detect Task Format, Not Reasoning Mode in Language Model Hidden States
This work challenges a common assumption in mechanistic interpretability that linear probes reveal distinct reasoning representations, highlighting the need for routine format deconfounding.
The authors show that linear probes of LLM hidden states achieve high accuracy in distinguishing reasoning types (deductive, inductive, abductive), but this is entirely due to format confounds (source identity, option count, response length) rather than genuine reasoning differences. After residualizing these confounds, probe accuracy drops to chance, and causal steering shows no functional link.
Linear probing of large language model (LLM) hidden states is widely used to claim that models learn distinct representations for different reasoning types. We test this by probing Qwen3-14B on three benchmarks spanning the classical trichotomy: LogiQA 2.0 (deductive), ARC-Challenge (inductive), and $α$NLI (abductive). At layer 32 of 40, linear probes achieve 100\% cross-validated accuracy with well-separated geometry (intrinsic dimensionalities: 20.6, 28.5, 33.6; convex hull contamination $\leq$1.5\%). However, this separation is entirely driven by format confounds. Residualizing source identity, option count, and response length reduces accuracy to chance. Trace-anchor similarity indicates largely shared reasoning across tasks (42.5\% agreement vs.\ 33.3\% chance), and causal steering with random controls ($n=20$) shows no functional link between geometry and reasoning mode ($p=0.286$). Thus, high probe accuracy reflects task format rather than computational structure, motivating routine format deconfounding in mechanistic interpretability.