Mamba Knockout for Unraveling Factual Information Flow
This provides interpretability insights for Mamba models, which are increasingly used as efficient alternatives to Transformers, though the approach is incremental as it adapts existing methods.
The paper investigates how factual information flows in Mamba state-space model language models by adapting Transformer attention interpretability techniques to Mamba-1 and Mamba-2, revealing patterns of information transmission and localization across tokens and layers.
This paper investigates the flow of factual information in Mamba State-Space Model (SSM)-based language models. We rely on theoretical and empirical connections to Transformer-based architectures and their attention mechanisms. Exploiting this relationship, we adapt attentional interpretability techniques originally developed for Transformers--specifically, the Attention Knockout methodology--to both Mamba-1 and Mamba-2. Using them we trace how information is transmitted and localized across tokens and layers, revealing patterns of subject-token information emergence and layer-wise dynamics. Notably, some phenomena vary between mamba models and Transformer based models, while others appear universally across all models inspected--hinting that these may be inherent to LLMs in general. By further leveraging Mamba's structured factorization, we disentangle how distinct "features" either enable token-to-token information exchange or enrich individual tokens, thus offering a unified lens to understand Mamba internal operations.