Multi-modal Conditional Bounding Box Regression for Music Score Following
This addresses the problem of automated music score following for musicians and musicologists, representing an incremental improvement over existing methods.
The paper tackles sheet-image-based audio-to-score alignment by proposing a conditional neural network that predicts matching positions in score images from audio, achieving new state-of-the-art results on a synthetic polyphonic piano benchmark and significantly improving alignment on real-world piano recordings with data augmentation.
This paper addresses the problem of sheet-image-based on-line audio-to-score alignment also known as score following. Drawing inspiration from object detection, a conditional neural network architecture is proposed that directly predicts x,y coordinates of the matching positions in a complete score sheet image at each point in time for a given musical performance. Experiments are conducted on a synthetic polyphonic piano benchmark dataset and the new method is compared to several existing approaches from the literature for sheet-image-based score following as well as an Optical Music Recognition baseline. The proposed approach achieves new state-of-the-art results and furthermore significantly improves the alignment performance on a set of real-world piano recordings by applying Impulse Responses as a data augmentation technique.