NeuralSound: Learning-based Modal Sound Synthesis With Acoustic Transfer
This work addresses the need for fast, high-quality sound synthesis in applications like virtual reality or gaming, though it appears incremental as it builds on existing modal analysis and acoustic transfer concepts.
The paper tackles the problem of slow modal sound synthesis by introducing a learning-based approach that combines a mixed vibration solver and an end-to-end sound radiation network, achieving synthesis in under one second on a GTX 3080 Ti GPU while maintaining high sound quality close to ground truth.
We present a novel learning-based modal sound synthesis approach that includes a mixed vibration solver for modal analysis and an end-to-end sound radiation network for acoustic transfer. Our mixed vibration solver consists of a 3D sparse convolution network and a Locally Optimal Block Preconditioned Conjugate Gradient module (LOBPCG) for iterative optimization. Moreover, we highlight the correlation between a standard modal vibration solver and our network architecture. Our radiation network predicts the Far-Field Acoustic Transfer maps (FFAT Maps) from the surface vibration of the object. The overall running time of our learning method for any new object is less than one second on a GTX 3080 Ti GPU while maintaining a high sound quality close to the ground truth that is computed using standard numerical methods. We also evaluate the numerical accuracy and perceptual accuracy of our sound synthesis approach on different objects corresponding to various materials.