Language Models Use Trigonometry to Do Addition
This provides the first representation-level explanation of LLMs' mathematical capabilities, addressing a lack of understanding in how they process simple tasks, which is incremental but specific to mathematical reasoning.
The authors reverse-engineered how three mid-sized LLMs compute addition, discovering that numbers are represented as a generalized helix and manipulated using a 'Clock' algorithm to perform arithmetic operations, with causal interventions verifying this representation-level explanation.
Mathematical reasoning is an increasingly important indicator of large language model (LLM) capabilities, yet we lack understanding of how LLMs process even simple mathematical tasks. To address this, we reverse engineer how three mid-sized LLMs compute addition. We first discover that numbers are represented in these LLMs as a generalized helix, which is strongly causally implicated for the tasks of addition and subtraction, and is also causally relevant for integer division, multiplication, and modular arithmetic. We then propose that LLMs compute addition by manipulating this generalized helix using the "Clock" algorithm: to solve $a+b$, the helices for $a$ and $b$ are manipulated to produce the $a+b$ answer helix which is then read out to model logits. We model influential MLP outputs, attention head outputs, and even individual neuron preactivations with these helices and verify our understanding with causal interventions. By demonstrating that LLMs represent numbers on a helix and manipulate this helix to perform addition, we present the first representation-level explanation of an LLM's mathematical capability.