How Pruning Reshapes Features: Sparse Autoencoder Analysis of Weight-Pruned Language Models
This addresses the problem of understanding compression effects on model interpretability for researchers and practitioners, though it is incremental as it builds on existing pruning and interpretability methods.
The study investigated how weight pruning affects the internal representations of language models, finding that rare features survive pruning far better than frequent ones, with correlations of rho = -1.0 in 11 of 17 conditions, and that Wanda pruning preserves feature structure up to 3.7x better than magnitude pruning.
Weight pruning is a standard technique for compressing large language models, yet its effect on learned internal representations remains poorly understood. We present the first systematic study of how unstructured pruning reshapes the feature geometry of language models, using Sparse Autoencoders (SAEs) as interpretability probes. Across three model families (Gemma 3 1B, Gemma 2 2B, Llama 3.2 1B), two pruning methods (magnitude and Wanda), and six sparsity levels (0--60%), we investigate five research questions spanning seed stability, feature survival, SAE transferability, feature fragility, and causal relevance. Our most striking finding is that rare SAE features--those with low firing rates--survive pruning far better than frequent ones, with within-condition Spearman correlations of rho = -1.0 in 11 of 17 experimental conditions. This counter-intuitive result suggests that pruning acts as implicit feature selection, preferentially destroying high-frequency generic features while preserving specialized rare ones. We further show that Wanda pruning preserves feature structure up to 3.7x better than magnitude pruning, that pre-trained SAEs remain viable on Wanda-pruned models up to 50% sparsity, and that geometric feature survival does not predict causal importance--a dissociation with implications for interpretability under compression.