Shift If You Can: Counting and Visualising Correction Operations for Beat Tracking Evaluation
This work addresses evaluation challenges for researchers in music information retrieval, but it is incremental as it builds on existing F-measure methods with a new operational perspective.
The paper tackles the problem of beat tracking evaluation by proposing a modified approach that frames evaluation in terms of the effort needed to align detected beats with ground truth, using a shifting operation over a larger tolerance window to replace insertions and deletions, and includes an annotation efficiency calculation and visualisation for qualitative assessment.
In this late-breaking abstract we propose a modified approach for beat tracking evaluation which poses the problem in terms of the effort required to transform a sequence of beat detections such that they maximise the well-known F-measure calculation when compared to a sequence of ground truth annotations. Central to our approach is the inclusion of a shifting operation conducted over an additional, larger, tolerance window, which can substitute the combination of insertions and deletions. We describe a straightforward calculation of annotation efficiency and combine this with an informative visualisation which can be of use for the qualitative evaluation of beat tracking systems. We make our implementation and visualisation code freely available in a GitHub repository.