Learning Robust Spatial Representations from Binaural Audio through Feature Distillation
This work addresses the challenge of robust spatial audio processing for applications like sound localization, though it is incremental as it builds on existing representation learning techniques.
The paper tackles the problem of learning spatial representations from binaural audio without labeled data by using feature distillation pretraining, resulting in improved direction-of-arrival estimation performance in noisy and reverberant environments compared to supervised and classic methods.
Recently, deep representation learning has shown strong performance in multiple audio tasks. However, its use for learning spatial representations from multichannel audio is underexplored. We investigate the use of a pretraining stage based on feature distillation to learn a robust spatial representation of binaural speech without the need for data labels. In this framework, spatial features are computed from clean binaural speech samples to form prediction labels. These clean features are then predicted from corresponding augmented speech using a neural network. After pretraining, we throw away the spatial feature predictor and use the learned encoder weights to initialize a DoA estimation model which we fine-tune for DoA estimation. Our experiments demonstrate that the pretrained models show improved performance in noisy and reverberant environments after fine-tuning for direction-of-arrival estimation, when compared to fully supervised models and classic signal processing methods.