Cross-Modal Music Retrieval and Applications: An Overview of Key Methodologies
This is an incremental overview of methodologies for cross-modal music retrieval, targeting users and researchers dealing with large digital music collections.
The paper addresses the challenge of cross-modal music retrieval, which involves finding corresponding information across different modalities like audio and sheet music, beyond exact audio identification as done by applications like Shazam.
There has been a rapid growth of digitally available music data, including audio recordings, digitized images of sheet music, album covers and liner notes, and video clips. This huge amount of data calls for retrieval strategies that allow users to explore large music collections in a convenient way. More precisely, there is a need for cross-modal retrieval algorithms that, given a query in one modality (e.g., a short audio excerpt), find corresponding information and entities in other modalities (e.g., the name of the piece and the sheet music). This goes beyond exact audio identification and subsequent retrieval of metainformation as performed by commercial applications like Shazam [1].