Why Do You Say It Like That? A Phoneme-Level Framework for Explainable Speech Deepfake Detection
This work addresses the need for interpretable AI in speech deepfake detection, offering a method to explain model decisions in human-understandable phonetic terms.
The paper introduces a phoneme-level explainability framework for speech deepfake detection that uses Grad-CAM and speech recognition to identify phonetic cues associated with spoofed speech. Experiments on ASVspoof 5 show comparable detection performance while providing interpretable linguistic insights.
As the accuracy of speech deepfake detection improves with the use of self-supervised representations such as wav2vec 2.0 and HuBERT, understanding why the speech is classified as bona fide or deepfake remains an open challenge. In pursuit of more trustworthy and interpretable artificial intelligence, we introduce a phoneme-level analysis framework that connects model predictions to measurable phonetic units. Our post-hoc explainability method is generally applicable to a variety of speech deepfake detection systems based on convolutional neural networks since it leverages Gradient-weighted Class Activation Mapping in conjunction with speech recognition to generate saliency maps aligned with phonemes and pauses. This pipeline reveals statistically significant attack- and speaker-dependent phonetic cues associated with spoofed speech in terms that humans can understand. Experiments using ASVspoof 5 show comparable detection performance to similar architectures while providing linguistic interpretations across speakers and spoofing conditions.