Understanding sparse autoencoder scaling in the presence of feature manifolds
This work addresses a potential bottleneck in scaling SAEs for neural network interpretability, but it is incremental as it builds on existing models and provides preliminary insights.
The authors investigated how feature manifolds affect the scaling behavior of sparse autoencoders (SAEs), finding that in one regime, SAEs learn far fewer features than the number of latents available, potentially limiting their effectiveness.
Sparse autoencoders (SAEs) model the activations of a neural network as linear combinations of sparsely occurring directions of variation (latents). The ability of SAEs to reconstruct activations follows scaling laws w.r.t. the number of latents. In this work, we adapt a capacity-allocation model from the neural scaling literature (Brill, 2024) to understand SAE scaling, and in particular, to understand how "feature manifolds" (multi-dimensional features) influence scaling behavior. Consistent with prior work, the model recovers distinct scaling regimes. Notably, in one regime, feature manifolds have the pathological effect of causing SAEs to learn far fewer features in data than there are latents in the SAE. We provide some preliminary discussion on whether or not SAEs are in this pathological regime in the wild.