Where MLLMs Attend and What They Rely On: Explaining Autoregressive Token Generation
This work addresses interpretability and reliability issues in MLLMs for researchers and practitioners, though it is incremental as it builds on existing attribution methods.
The authors tackled the problem of understanding how multimodal large language models (MLLMs) rely on visual inputs during token generation, and they introduced EAGLE, a lightweight black-box framework that outperforms existing methods in faithfulness, localization, and hallucination diagnosis while using less GPU memory.
Multimodal large language models (MLLMs) have demonstrated remarkable capabilities in aligning visual inputs with natural language outputs. Yet, the extent to which generated tokens depend on visual modalities remains poorly understood, limiting interpretability and reliability. In this work, we present EAGLE, a lightweight black-box framework for explaining autoregressive token generation in MLLMs. EAGLE attributes any selected tokens to compact perceptual regions while quantifying the relative influence of language priors and perceptual evidence. The framework introduces an objective function that unifies sufficiency (insight score) and indispensability (necessity score), optimized via greedy search over sparsified image regions for faithful and efficient attribution. Beyond spatial attribution, EAGLE performs modality-aware analysis that disentangles what tokens rely on, providing fine-grained interpretability of model decisions. Extensive experiments across open-source MLLMs show that EAGLE consistently outperforms existing methods in faithfulness, localization, and hallucination diagnosis, while requiring substantially less GPU memory. These results highlight its effectiveness and practicality for advancing the interpretability of MLLMs. The code will be released at https://ruoyuchen10.github.io/EAGLE/.