Sounderfeit: Cloning a Physical Model using a Conditional Adversarial Autoencoder
This work addresses the challenge of creating sound synthesizers from data for audio engineering and music production, but it is incremental as it builds on existing adversarial autoencoder methods.
The researchers tackled the problem of cloning physical models of bowed string synthesizers by using a conditional adversarial autoencoder for parameter estimation and resynthesis, resulting in a real-time synthesis system that can copy parameter-signal relationships from recorded data.
An adversarial autoencoder conditioned on known parameters of a physical modeling bowed string synthesizer is evaluated for use in parameter estimation and resynthesis tasks. Latent dimensions are provided to capture variance not explained by the conditional parameters. Results are compared with and without the adversarial training, and a system capable of "copying" a given parameter-signal bidirectional relationship is examined. A real-time synthesis system built on a generative, conditioned and regularized neural network is presented, allowing to construct engaging sound synthesizers based purely on recorded data.