Lyric document embeddings for music tagging
This work addresses music tagging for applications like content organization, but it is incremental as it compares existing methods rather than introducing new ones.
The study tackled the problem of embedding song lyrics for music tagging by comparing five token-level and four document-level representation methods on tens of millions of songs, finding that simple averaging of word embeddings outperformed more complex neural architectures in tasks like genre classification and era detection.
We present an empirical study on embedding the lyrics of a song into a fixed-dimensional feature for the purpose of music tagging. Five methods of computing token-level and four methods of computing document-level representations are trained on an industrial-scale dataset of tens of millions of songs. We compare simple averaging of pretrained embeddings to modern recurrent and attention-based neural architectures. Evaluating on a wide range of tagging tasks such as genre classification, explicit content identification and era detection, we find that averaging word embeddings outperform more complex architectures in many downstream metrics.