Cross-Attention is Half Explanation in Speech-to-Text Models
This work addresses a gap in interpreting cross-attention for speech-to-text models, showing its limitations for applications like timestamp estimation, though it is incremental as it builds on debates from NLP.
The study assessed the explanatory power of cross-attention in speech-to-text models by comparing its scores to saliency maps, finding that cross-attention captures only 50% of input relevance and aligns with 52-75% of saliency, revealing it as an incomplete proxy for model predictions.
Cross-attention is a core mechanism in encoder-decoder architectures, widespread in many fields, including speech-to-text (S2T) processing. Its scores have been repurposed for various downstream applications--such as timestamp estimation and audio-text alignment--under the assumption that they reflect the dependencies between input speech representation and the generated text. While the explanatory nature of attention mechanisms has been widely debated in the broader NLP literature, this assumption remains largely unexplored within the speech domain. To address this gap, we assess the explanatory power of cross-attention in S2T models by comparing its scores to input saliency maps derived from feature attribution. Our analysis spans monolingual and multilingual, single-task and multi-task models at multiple scales, and shows that attention scores moderately to strongly align with saliency-based explanations, particularly when aggregated across heads and layers. However, it also shows that cross-attention captures only about 50% of the input relevance and, in the best case, only partially reflects how the decoder attends to the encoder's representations--accounting for just 52-75% of the saliency. These findings uncover fundamental limitations in interpreting cross-attention as an explanatory proxy, suggesting that it offers an informative yet incomplete view of the factors driving predictions in S2T models.