Real-valued parametric conditioning of an RNN for interactive sound synthesis
This work addresses interactive sound synthesis for musicians or audio engineers, but it appears incremental as it focuses on applying existing conditioning techniques to RNNs for audio.
The paper tackles the problem of conditioning audio synthesis models with real-valued parameters, resulting in an RNN-based synthesizer that can be controlled like a musical instrument with continuous parameters, and it demonstrates generalization and responsiveness to unseen parameter values and sequences.
A Recurrent Neural Network (RNN) for audio synthesis is trained by augmenting the audio input with information about signal characteristics such as pitch, amplitude, and instrument. The result after training is an audio synthesizer that is played like a musical instrument with the desired musical characteristics provided as continuous parametric control. The focus of this paper is on conditioning data-driven synthesis models with real-valued parameters, and in particular, on the ability of the system a) to generalize and b) to be responsive to parameter values and sequences not seen during training.