Interpreting Graphic Notation with MusicLDM: An AI Improvisation of Cornelius Cardew's Treatise
This work addresses the challenge of performing and interpreting graphic scores for contemporary/experimental musicians, offering an incremental AI-based method to automate and innovate composition.
The authors tackled the problem of interpreting graphic notation for music composition by using ChatGPT to convert visual elements of Cornelius Cardew's Treatise into text prompts, which were then fed into MusicLDM to generate music, resulting in a seamless composition through an outpainting technique. This approach demonstrated AI's ability to transform visual stimuli into sound, expanding creative possibilities in experimental music.
This work presents a novel method for composing and improvising music inspired by Cornelius Cardew's Treatise, using AI to bridge graphic notation and musical expression. By leveraging OpenAI's ChatGPT to interpret the abstract visual elements of Treatise, we convert these graphical images into descriptive textual prompts. These prompts are then input into MusicLDM, a pre-trained latent diffusion model designed for music generation. We introduce a technique called "outpainting," which overlaps sections of AI-generated music to create a seamless and cohesive composition. We demostrate a new perspective on performing and interpreting graphic scores, showing how AI can transform visual stimuli into sound and expand the creative possibilities in contemporary/experimental music composition. Musical pieces are available at https://bit.ly/TreatiseAI