Beyond Decodability: Reconstructing Language Model Representations with an Encoding Probe
For researchers studying interpretability of language models, this provides a complementary method to decoding probes, but the contribution is incremental as it builds on existing probing frameworks.
The authors propose an Encoding Probe that reconstructs language model representations from interpretable features, addressing limitations of decoding probes. Results show speaker effects vary across training objectives, while syntax and lexicon contribute independently.
Probing is widely used to study which features can be decoded from language model representations. However, the common decoding probe approach has two limitations that we aim to solve with our new encoding probe approach: contributions of different features to model representations cannot be directly compared, and feature correlations can affect probing results. We present an Encoding Probe that reverses this direction and reconstructs internal representations of models using interpretable features. We evaluate this method on text and speech transformer models, using feature sets spanning acoustics, phonetics, syntax, lexicon, and speaker identity. Our results suggest that speaker-related effects vary strongly across different training objectives and datasets, while syntactic and lexical features contribute independently to reconstruction. These results show that the Encoding Probe provides a complementary perspective on interpreting model representations beyond decodability.