BumbleBee: A Transformer for Music
This addresses the challenge of long sequence modeling in music generation for AI researchers, but it appears incremental as it builds on existing transformer and longformer techniques.
The authors tackled the problem of applying transformers to long sequences in music generation by introducing BumbleBee, a transformer model using dilating sliding windows for attention, and benchmarked it against existing models like the music transformer and LSTM on the JSB Chorales dataset, but no concrete results or numbers were provided.
We will introduce BumbleBee, a transformer model that will generate MIDI music data . We will tackle the issue of transformers applied to long sequences by implementing a longformer generative model that uses dilating sliding windows to compute the attention layers. We will compare our results to that of the music transformer and Long-Short term memory (LSTM) to benchmark our results. This analysis will be performed using piano MIDI files, in particular , the JSB Chorales dataset that has already been used for other research works (Huang et al., 2018)