Fully Automatic Page Turning on Real Scores
This addresses a practical issue for musicians performing with sheet music, but it is incremental as it builds on existing multi-modal neural network approaches.
The researchers tackled the problem of automatic page turning for musicians by developing a system that directly uses sheet images and audio to predict positions, achieving a proof-of-concept integration with a physical page-turning machine.
We present a prototype of an automatic page turning system that works directly on real scores, i.e., sheet images, without any symbolic representation. Our system is based on a multi-modal neural network architecture that observes a complete sheet image page as input, listens to an incoming musical performance, and predicts the corresponding position in the image. Using the position estimation of our system, we use a simple heuristic to trigger a page turning event once a certain location within the sheet image is reached. As a proof of concept we further combine our system with an actual machine that will physically turn the page on command.