WavShape: Information-Theoretic Speech Representation Learning for Fair and Privacy-Aware Audio Processing
This work addresses fairness and privacy issues in audio processing for applications like speech systems, though it is incremental as it builds on existing information-theoretic and self-supervised methods.
The paper tackles the problem of speech embeddings retaining sensitive attributes like speaker identity, which can lead to bias and privacy risks, by proposing WavShape, a framework that reduces mutual information with sensitive attributes by up to 81% while retaining 97% of task-relevant information.
Speech embeddings often retain sensitive attributes such as speaker identity, accent, or demographic information, posing risks in biased model training and privacy leakage. We propose WavShape, an information-theoretic speech representation learning framework that optimizes embeddings for fairness and privacy while preserving task-relevant information. We leverage mutual information (MI) estimation using the Donsker-Varadhan formulation to guide an MI-based encoder that systematically filters sensitive attributes while maintaining speech content essential for downstream tasks. Experimental results on three known datasets show that WavShape reduces MI between embeddings and sensitive attributes by up to 81% while retaining 97% of task-relevant information. By integrating information theory with self-supervised speech models, this work advances the development of fair, privacy-aware, and resource-efficient speech systems.