Explaining Spectrograms in Machine Learning: A Study on Neural Networks for Speech Classification
This work enhances explainability in speech classification for researchers and practitioners, though it is incremental as it applies existing methods to a specific domain.
The study investigated discriminative patterns in neural networks for vowel classification, identifying key frequencies through class activation mapping and comparing them with linguistic knowledge, which provided insights into misclassifications and acoustic cues.
This study investigates discriminative patterns learned by neural networks for accurate speech classification, with a specific focus on vowel classification tasks. By examining the activations and features of neural networks for vowel classification, we gain insights into what the networks "see" in spectrograms. Through the use of class activation mapping, we identify the frequencies that contribute to vowel classification and compare these findings with linguistic knowledge. Experiments on a American English dataset of vowels showcases the explainability of neural networks and provides valuable insights into the causes of misclassifications and their characteristics when differentiating them from unvoiced speech. This study not only enhances our understanding of the underlying acoustic cues in vowel classification but also offers opportunities for improving speech recognition by bridging the gap between abstract representations in neural networks and established linguistic knowledge