Binaural Rendering of Ambisonic Signals by Neural Networks
This work addresses the problem of immersive audio rendering for virtual reality applications, but it is incremental as it builds on existing deep learning approaches with specific dataset and framework improvements.
The paper tackles binaural rendering of ambisonic signals for virtual reality by proposing a deep learning framework that outperforms conventional methods in objective metrics and achieves comparable subjective metrics, with results including an SDR of 7.32 and MOSs ranging from 3.58 to 3.87 across quality, timbre, localization, and immersion dimensions.
Binaural rendering of ambisonic signals is of broad interest to virtual reality and immersive media. Conventional methods often require manually measured Head-Related Transfer Functions (HRTFs). To address this issue, we collect a paired ambisonic-binaural dataset and propose a deep learning framework in an end-to-end manner. Experimental results show that neural networks outperform the conventional method in objective metrics and achieve comparable subjective metrics. To validate the proposed framework, we experimentally explore different settings of the input features, model structures, output features, and loss functions. Our proposed system achieves an SDR of 7.32 and MOSs of 3.83, 3.58, 3.87, 3.58 in quality, timbre, localization, and immersion dimensions.