Self-supervised Audio Spatialization with Correspondence Classifier
This addresses the need for accessible spatial audio creation for general audiences, but appears incremental as it builds on existing self-supervised and classification techniques.
The paper tackles the problem of generating spatial audio from video and monaural audio using a self-supervised network with an auxiliary classifier, achieving effective results as validated on a large-scale dataset.
Spatial audio is an essential medium to audiences for 3D visual and auditory experience. However, the recording devices and techniques are expensive or inaccessible to the general public. In this work, we propose a self-supervised audio spatialization network that can generate spatial audio given the corresponding video and monaural audio. To enhance spatialization performance, we use an auxiliary classifier to classify ground-truth videos and those with audio where the left and right channels are swapped. We collect a large-scale video dataset with spatial audio to validate the proposed method. Experimental results demonstrate the effectiveness of the proposed model on the audio spatialization task.