A Computational Evaluation of Musical Pattern Discovery Algorithms
This work addresses the problem of algorithm evaluation for researchers in music informatics, but it is incremental as it builds on existing methods without introducing a new paradigm.
The paper tackled the challenge of evaluating musical pattern discovery algorithms by introducing three computational methods—Pattern Polling, Comparative Classification, and Synthetic Data—to analyze and compare algorithm outputs, revealing that rhythmic features contribute most to discrepancies between algorithms and human annotations.
Pattern discovery algorithms in the music domain aim to find meaningful components in musical compositions. Over the years, although many algorithms have been developed for pattern discovery in music data, it remains a challenging task. To gain more insight into the efficacy of these algorithms, we introduce three computational methods for examining their output: Pattern Polling, to combine the patterns; Comparative Classification, to differentiate the patterns; Synthetic Data, to inject predetermined patterns. In combining and differentiating the patterns extracted by algorithms, we expose how they differ from the patterns annotated by humans as well as between algorithms themselves, with rhythmic features contributing the most to the algorithm-human and algorithm-algorithm discrepancies. Despite the difficulty in reconciling and evaluating the divergent patterns extracted from algorithms, we identify some possibilities for addressing them. In particular, we generate controllable synthesised data with predetermined patterns planted into random data, thereby leaving us better able to inspect, compare, validate, and select the algorithms. We provide a concrete example of synthesising data for understanding the algorithms and expand our discussion to the potential and limitations of such an approach.