Improving Real-time Score Following in Opera by Combining Music with Lyrics Tracking
This work addresses the problem of real-time score following in opera for applications in music technology, but it is incremental as it builds on existing trackers.
The paper tackles the challenge of fully automatic opera tracking by proposing a pipeline that combines a music tracker for orchestral parts with a lyrics tracker to correct for text-dominant sections, demonstrating improved accuracy and robustness on the opera Don Giovanni.
Fully automatic opera tracking is challenging because of the acoustic complexity of the genre, combining musical and linguistic information (singing, speech) in complex ways. In this paper, we propose a new pipeline for complete opera tracking. The pipeline is based on two trackers. A music tracker that has proven to be effective at tracking orchestral parts, will lead the tracking process. In addition, a lyrics tracker, that has recently been shown to reliably track the lyrics of opera songs, will correct the music tracker when tracking parts that have a text dominance over the music. We will demonstrate the efficiency of this method on the opera Don Giovanni, showing that this technique helps improving accuracy and robustness of a complete opera tracker.