AudioLens: A Closer Look at Auditory Attribute Perception of Large Audio-Language Models
This provides insights into auditory attribute processing for researchers working on interpretability and performance improvement of audio-language models, though it is incremental as it builds on existing models.
This work analyzed how large audio-language models internally perceive auditory attributes, finding that attribute information decreases with layer depth when recognition fails and earlier resolution correlates with better accuracy, and demonstrated a method to enhance these models.
Understanding the internal mechanisms of large audio-language models (LALMs) is crucial for interpreting their behavior and improving performance. This work presents the first in-depth analysis of how LALMs internally perceive and recognize auditory attributes. By applying vocabulary projection on three state-of-the-art LALMs, we track how attribute information evolves across layers and token positions. We find that attribute information generally decreases with layer depth when recognition fails, and that resolving attributes at earlier layers correlates with better accuracy. Moreover, LALMs heavily rely on querying auditory inputs for predicting attributes instead of aggregating necessary information in hidden states at attribute-mentioning positions. Based on our findings, we demonstrate a method to enhance LALMs. Our results offer insights into auditory attribute processing, paving the way for future improvements.