Cross entropy as objective function for music generative models
This work addresses the selection of objective functions for music generation models, which is an incremental improvement in a domain-specific area of AI.
The paper investigates whether cross entropy is a suitable objective function for training music generative models by correlating loss improvements with reduced randomness and consistent melody generation, finding that cross entropy effectively reduces randomness but struggles with long-term consistency.
The election of the function to optimize when training a machine learning model is very important since this is which lets the model learn. It is not trivial since there are many options, each for different purposes. In the case of sequence generation of text, cross entropy is a common option because of its capability to quantify the predictive behavior of the model. In this paper, we test the validity of cross entropy for a music generator model with an experiment that aims to correlate improvements in the loss value with the reduction of randomness and the ability to keep consistent melodies. We also analyze the relationship between these two aspects which respectively relate to short and long term memory and how they behave and are learned differently.