Rigid-Body Sound Synthesis with Differentiable Modal Resonators
This work addresses sound synthesis for applications like virtual environments and music production by offering a more efficient alternative to traditional methods, though it is incremental as it builds on deep learning approaches.
The authors tackled the problem of synthesizing rigid-body sounds by introducing an end-to-end deep learning framework that generates modal resonators for 2D shapes and materials using differentiable IIR filters, achieving results that enable physically-informed synthesis from audio recordings.
Physical models of rigid bodies are used for sound synthesis in applications from virtual environments to music production. Traditional methods such as modal synthesis often rely on computationally expensive numerical solvers, while recent deep learning approaches are limited by post-processing of their results. In this work we present a novel end-to-end framework for training a deep neural network to generate modal resonators for a given 2D shape and material, using a bank of differentiable IIR filters. We demonstrate our method on a dataset of synthetic objects, but train our model using an audio-domain objective, paving the way for physically-informed synthesisers to be learned directly from recordings of real-world objects.