A Survey on Music Generation from Single-Modal, Cross-Modal, and Multi-Modal Perspectives
It provides a survey for researchers in AI and music, but it is incremental as it summarizes existing work without new results.
This paper reviews the field of multi-modal music generation, categorizing systems based on modalities and discussing representation, alignment, and guidance, while identifying challenges like integration and datasets.
Multi-modal music generation, using multiple modalities like text, images, and video alongside musical scores and audio as guidance, is an emerging research area with broad applications. This paper reviews this field, categorizing music generation systems from the perspective of modalities. The review covers modality representation, multi-modal data alignment, and their utilization to guide music generation. Current datasets and evaluation methods are also discussed. Key challenges in this area include effective multi-modal integration, large-scale comprehensive datasets, and systematic evaluation methods. Finally, an outlook on future research directions is provided, focusing on creativity, efficiency, multi-modal alignment, and evaluation.