SIGMA: Saliency-Guided Sparse Mask Attacks for Speech Emotion Recognition
For researchers studying adversarial robustness in SER, this work provides an interpretable and efficient attack framework, though it is incremental as it combines existing techniques (saliency maps, sparse attacks) without major breakthroughs.
The paper proposes SIGMA, a saliency-guided sparse mask attack for speech emotion recognition (SER) that uses post-hoc XAI to identify perturbation masks, achieving competitive attack success rates while maintaining explanation consistency across IEMOCAP and TESS datasets.
Speech conveys rich emotional information. As Speech Emotion Recognition (SER) is usually deployed in privacy-sensitive and reliability-critical environments, adversarial attacks on SER have attracted increasing attention. Existing sparse attacks control the number of perturbed elements, yet, they often lack explainability guidance and explicit measures of explanation consistency. A unified treatment of sparsity and magnitude constraints is also uncommon. In addition, transferability across attack families and target models remains limited. Hence, we propose a SalIency-Guided sparse Mask Attack (SIGMA). On self-supervised speech features, we use post-hoc explainable artificial intelligence (XAI) techniques to produce saliency maps and identify the scope of the mask, and then restrict magnitude-bounded updates to this mask. The mask is computed once and can be reused across models and different sparsity attacks to amortise cost. We evaluate on the IEMOCAP and TESS datasets. Under matched budgets and across multiple sparse-attack settings, SIGMA maintains competitive attack success rates, navigating a conscious trade-off between attack efficacy and explanation consistency. SIGMA therefore provides an efficient and interpretable framework for analysing the vulnerability and explanation behaviour of SER models under structured perturbations.