Composer's Assistant 2: Interactive Multi-Track MIDI Infilling with Fine-Grained User Control
This work provides musicians with enhanced control for multi-track MIDI infilling in digital audio workstations, though it is incremental as it builds on an existing system.
The paper tackles the problem of interactive human-computer music composition by upgrading the Composer's Assistant system with fine-grained user controls, such as rhythmic and pitch conditioning, achieving a dramatic improvement in objective metrics and no significant difference from real music in a listening study.
We introduce Composer's Assistant 2, a system for interactive human-computer composition in the REAPER digital audio workstation. Our work upgrades the Composer's Assistant system (which performs multi-track infilling of symbolic music at the track-measure level) with a wide range of new controls to give users fine-grained control over the system's outputs. Controls introduced in this work include two types of rhythmic conditioning controls, horizontal and vertical note onset density controls, several types of pitch controls, and a rhythmic interest control. We train a T5-like transformer model to implement these controls and to serve as the backbone of our system. With these controls, we achieve a dramatic improvement in objective metrics over the original system. We also study how well our model understands the meaning of our controls, and we conduct a listening study that does not find a significant difference between real music and music composed in a co-creative fashion with our system. We release our complete system, consisting of source code, pretrained models, and REAPER scripts.