Language Models are Drummers: Drum Composition with Natural Language Pre-Training
This addresses the problem of data scarcity in automatic music generation for niche domains, offering an incremental transfer learning method.
The paper tackles the challenge of generating music for less common genres and instruments with limited data by finetuning GPT3, a large language model pre-trained on text, on only hundreds of drum MIDI files, showing it can produce reasonable drum grooves while non-pre-trained models fail. They propose a tailored evaluation method and compare GPT3-generated grooves to human performances, finding the approach viable and promising.
Automatic music generation with artificial intelligence typically requires a large amount of data which is hard to obtain for many less common genres and musical instruments. To tackle this issue, we present ongoing work and preliminary findings on the possibility for deep models to transfer knowledge from language to music, by finetuning large language models pre-trained on a massive text corpus on only hundreds of MIDI files of drum performances. We show that by doing so, one of the largest, state-of-the-art models (GPT3) is capable of generating reasonable drum grooves, while models that are not pre-trained (Transformer) shows no such ability beyond naive repetition. Evaluating generated music is a challenging task, more so is evaluating drum grooves with little precedence in literature. Hence, we propose a tailored structural evaluation method and analyze drum grooves produced by GPT3 compared to those played by human professionals, exposing the strengths and weaknesses of such generation by language-to-music transfer. Our findings suggest that language-to-music transfer learning with large language models is viable and promising.