Resounding Acoustic Fields with Reciprocity
This work addresses the need for flexible sound modeling in virtual environments to support dynamic source positions, offering a novel approach to acoustic field learning.
The paper tackles the problem of estimating room impulse responses at arbitrary emitter positions from sparse measurements, introducing a task called resounding. It proposes Versa, a physics-inspired method that uses reciprocity and self-supervised learning, showing substantial performance improvements on simulated and real-world datasets and enhancing immersive spatial sound experiences in perceptual studies.
Achieving immersive auditory experiences in virtual environments requires flexible sound modeling that supports dynamic source positions. In this paper, we introduce a task called resounding, which aims to estimate room impulse responses at arbitrary emitter location from a sparse set of measured emitter positions, analogous to the relighting problem in vision. We leverage the reciprocity property and introduce Versa, a physics-inspired approach to facilitating acoustic field learning. Our method creates physically valid samples with dense virtual emitter positions by exchanging emitter and listener poses. We also identify challenges in deploying reciprocity due to emitter/listener gain patterns and propose a self-supervised learning approach to address them. Results show that Versa substantially improve the performance of acoustic field learning on both simulated and real-world datasets across different metrics. Perceptual user studies show that Versa can greatly improve the immersive spatial sound experience. Code, dataset and demo videos are available on the project website: https://waves.seas.upenn.edu/projects/versa.