Deep Music Analogy Via Latent Representation Disentanglement
This addresses the challenge of automated music analogy-making for creative AI applications, but it is incremental as it builds on existing disentanglement methods for a specific domain.
The paper tackled the problem of generating creative music analogies by disentangling pitch and rhythm representations in 8-beat music clips conditioned on chords, using an explicitly-constrained variational autoencoder (EC^2-VAE) as a unified solution, and validated it with objective measurements and subjective evaluation.
Analogy-making is a key method for computer algorithms to generate both natural and creative music pieces. In general, an analogy is made by partially transferring the music abstractions, i.e., high-level representations and their relationships, from one piece to another; however, this procedure requires disentangling music representations, which usually takes little effort for musicians but is non-trivial for computers. Three sub-problems arise: extracting latent representations from the observation, disentangling the representations so that each part has a unique semantic interpretation, and mapping the latent representations back to actual music. In this paper, we contribute an explicitly-constrained variational autoencoder (EC$^2$-VAE) as a unified solution to all three sub-problems. We focus on disentangling the pitch and rhythm representations of 8-beat music clips conditioned on chords. In producing music analogies, this model helps us to realize the imaginary situation of "what if" a piece is composed using a different pitch contour, rhythm pattern, or chord progression by borrowing the representations from other pieces. Finally, we validate the proposed disentanglement method using objective measurements and evaluate the analogy examples by a subjective study.