R-Spin: Efficient Speaker and Noise-invariant Representation Learning with Acoustic Pieces
This addresses the challenge of robust speech processing in diverse acoustic environments for applications like speech recognition, with incremental improvements over existing methods.
The paper tackles the problem of learning speaker and noise-invariant speech representations by introducing R-Spin, a data-efficient self-supervision method that predicts acoustic pieces, resulting in a 12X reduction in computational resources and outperforming previous state-of-the-art methods in severely distorted speech scenarios.
This paper introduces Robust Spin (R-Spin), a data-efficient domain-specific self-supervision method for speaker and noise-invariant speech representations by learning discrete acoustic units with speaker-invariant clustering (Spin). R-Spin resolves Spin's issues and enhances content representations by learning to predict acoustic pieces. R-Spin offers a 12X reduction in computational resources compared to previous state-of-the-art methods while outperforming them in severely distorted speech scenarios. This paper provides detailed analyses to show how discrete units contribute to speech encoder training and improving robustness in diverse acoustic environments.