AudioMNIST: Exploring Explainable Artificial Intelligence for Audio Analysis on a Simple Benchmark
This work addresses the need for explainable AI in audio analysis, though it is incremental as it applies existing XAI methods to a new dataset and domain.
The paper tackled the problem of understanding feature selection in deep neural networks for audio classification by introducing a new dataset of 30,000 spoken digit samples and applying Layer-wise Relevance Propagation (LRP) to generate explanations, demonstrating that audible heatmaps improved interpretability over visual ones in a user study.
Explainable Artificial Intelligence (XAI) is targeted at understanding how models perform feature selection and derive their classification decisions. This paper explores post-hoc explanations for deep neural networks in the audio domain. Notably, we present a novel Open Source audio dataset consisting of 30,000 audio samples of English spoken digits which we use for classification tasks on spoken digits and speakers' biological sex. We use the popular XAI technique Layer-wise Relevance Propagation (LRP) to identify relevant features for two neural network architectures that process either waveform or spectrogram representations of the data. Based on the relevance scores obtained from LRP, hypotheses about the neural networks' feature selection are derived and subsequently tested through systematic manipulations of the input data. Further, we take a step beyond visual explanations and introduce audible heatmaps. We demonstrate the superior interpretability of audible explanations over visual ones in a human user study.