Investigating Faithfulness in Large Audio Language Models
This addresses faithfulness for safety-sensitive applications in audio-language AI, but is incremental as it extends prior text-based work to a new modality.
The study investigated whether chain-of-thought (CoT) representations in large audio-language models (LALMs) are faithful to their decision processes, finding that LALMs generally produce CoTs that appear faithful after interventions on datasets like SAKURA and MMAR.
Faithfulness measures whether chain-of-thought (CoT) representations accurately reflect a model's decision process and can be used as reliable explanations. Prior work has shown that CoTs from text-based LLMs are often unfaithful. This question has not been explored for large audio-language models (LALMs), where faithfulness is critical for safety-sensitive applications. Reasoning in LALMs is also more challenging, as models must first extract relevant clues from audio before reasoning over them. In this paper, we investigate the faithfulness of CoTs produced by several LALMs by applying targeted interventions, including paraphrasing, filler token injection, early answering, and introducing mistakes, on two challenging reasoning datasets: SAKURA and MMAR. After going through the aforementioned interventions across several datasets and tasks, our experiments suggest that, LALMs generally produce CoTs that appear to be faithful to their underlying decision processes.