Inspecting and Interacting with Meaningful Music Representations using VAE
This addresses the need for more intuitive control in music composition tools, though it is incremental as it builds on existing VAE methods.
The paper tackles the problem of controlling music generation with VAEs by making latent representations interpretable for pitch and rhythm, enabling users to modify these features via a graphical interface.
Variational Autoencoders(VAEs) have already achieved great results on image generation and recently made promising progress on music generation. However, the generation process is still quite difficult to control in the sense that the learned latent representations lack meaningful music semantics. It would be much more useful if people can modify certain music features, such as rhythm and pitch contour, via latent representations to test different composition ideas. In this paper, we propose a new method to inspect the pitch and rhythm interpretations of the latent representations and we name it disentanglement by augmentation. Based on the interpretable representations, an intuitive graphical user interface is designed for users to better direct the music creation process by manipulating the pitch contours and rhythmic complexity.