AccoMontage: Accompaniment Arrangement via Phrase Selection and Style Transfer
This addresses the challenge of long-term music generation for composers and musicians, though it is incremental as it builds on existing methods in a hybrid approach.
The authors tackled the problem of generating piano accompaniments for folk/pop songs from lead sheets by proposing AccoMontage, a system that combines phrase selection and style transfer, resulting in significantly outperforming baselines with well-structured and textured outputs.
Accompaniment arrangement is a difficult music generation task involving intertwined constraints of melody, harmony, texture, and music structure. Existing models are not yet able to capture all these constraints effectively, especially for long-term music generation. To address this problem, we propose AccoMontage, an accompaniment arrangement system for whole pieces of music through unifying phrase selection and neural style transfer. We focus on generating piano accompaniments for folk/pop songs based on a lead sheet (i.e., melody with chord progression). Specifically, AccoMontage first retrieves phrase montages from a database while recombining them structurally using dynamic programming. Second, chords of the retrieved phrases are manipulated to match the lead sheet via style transfer. Lastly, the system offers controls over the generation process. In contrast to pure learning-based approaches, AccoMontage introduces a novel hybrid pathway, in which rule-based optimization and deep learning are both leveraged to complement each other for high-quality generation. Experiments show that our model generates well-structured accompaniment with delicate texture, significantly outperforming the baselines.