Live Score Following on Sheet Music Images
This addresses the challenge of real-time music synchronization for musicians or applications, but it is incremental as it builds on existing score-following methods with a new modality.
The paper tackles the problem of live score following by matching audio streams directly to sheet music images using a multi-modal convolutional neural network, achieving a proof-of-concept implementation tested on 3 simple piano pieces.
In this demo we show a novel approach to score following. Instead of relying on some symbolic representation, we are using a multi-modal convolutional neural network to match the incoming audio stream directly to sheet music images. This approach is in an early stage and should be seen as proof of concept. Nonetheless, the audience will have the opportunity to test our implementation themselves via 3 simple piano pieces.