Exploring Musical Roots: Applying Audio Embeddings to Empower Influence Attribution for a Generative Music Model
This work addresses the issue of inadvertent appropriation in generative music for model creators and users, representing an incremental step toward automated influence attribution.
The paper tackles the problem of identifying training data attribution in generative music models by developing a methodology using audio embeddings (CLMR and CLAP) to measure similarity across 5 million audio clips, validated with a human study, showing how modifications like pitch shifting affect similarity.
Every artist has a creative process that draws inspiration from previous artists and their works. Today, "inspiration" has been automated by generative music models. The black box nature of these models obscures the identity of the works that influence their creative output. As a result, users may inadvertently appropriate, misuse, or copy existing artists' works. We establish a replicable methodology to systematically identify similar pieces of music audio in a manner that is useful for understanding training data attribution. A key aspect of our approach is to harness an effective music audio similarity measure. We compare the effect of applying CLMR and CLAP embeddings to similarity measurement in a set of 5 million audio clips used to train VampNet, a recent open source generative music model. We validate this approach with a human listening study. We also explore the effect that modifications of an audio example (e.g., pitch shifting, time stretching, background noise) have on similarity measurements. This work is foundational to incorporating automated influence attribution into generative modeling, which promises to let model creators and users move from ignorant appropriation to informed creation. Audio samples that accompany this paper are available at https://tinyurl.com/exploring-musical-roots.