The Rate-Distortion-Polysemanticity Tradeoff in SAEs
For researchers in mechanistic interpretability, this work provides a theoretical framework explaining why SAEs often fail to learn monosemantic features, highlighting that polysemanticity is a data-driven problem rather than solely an architectural one.
The paper characterizes a fundamental tradeoff between distortion (reconstruction accuracy), rate (feature efficiency), and polysemanticity (interpretability) in Sparse Autoencoders (SAEs). The authors theoretically and empirically show that enforcing monosemanticity necessarily increases rate and distortion, and that the degree of polysemanticity in optimal SAEs is determined by the co-occurrence probability of features in the training data.
Sparse Autoencoders (SAEs) that can accurately reconstruct their input (minimizing distortion) by making efficient use of few features (minimizing the rate) often fail to learn monosemantic representations (highly interpretable), limiting their usefulness for mechanistic interpretability. In this paper, we characterise this tension in learning faithful, efficient, and interpretable explanations, introducing the Rate-Distortion-Polysemanticity tradeoff in SAEs. Under toy-modeling assumptions, we theoretically and empirically show that restricting the SAE to be monosemantic necessarily comes with an increase in rate and distortion. Assuming a generative model behind the input observations, we further demonstrate that the degree of polysemanticity of optimal SAEs is determined by the training data distribution, especially by the probability of features to co-occur. Finally, we extend the analysis to real-world settings by deriving necessary conditions that a polysemanticity measure should satisfy when the data-generating process is unknown, and we benchmark existing proxy metrics on SAEs trained on Large Language Models. Taken together, our findings show that polysemanticity is a data problem that should be accounted for when addressing it at the architectural and optimization level.