Monotonic Representation of Numeric Properties in Language Models
This addresses the interpretability of language models for researchers and practitioners, though it is incremental as it builds on existing work in representation editing.
The paper tackles the problem of understanding how language models encode numeric properties like birth years, finding low-dimensional subspaces that represent these properties monotonically and allowing editable adjustments to model outputs, such as changing Karl Popper's birth year from 1902 to later dates like 1968.
Language models (LMs) can express factual knowledge involving numeric properties such as Karl Popper was born in 1902. However, how this information is encoded in the model's internal representations is not understood well. Here, we introduce a simple method for finding and editing representations of numeric properties such as an entity's birth year. Empirically, we find low-dimensional subspaces that encode numeric properties monotonically, in an interpretable and editable fashion. When editing representations along directions in these subspaces, LM output changes accordingly. For example, by patching activations along a "birthyear" direction we can make the LM express an increasingly late birthyear: Karl Popper was born in 1929, Karl Popper was born in 1957, Karl Popper was born in 1968. Property-encoding directions exist across several numeric properties in all models under consideration, suggesting the possibility that monotonic representation of numeric properties consistently emerges during LM pretraining. Code: https://github.com/bheinzerling/numeric-property-repr