Quantitative Evaluation of Motif Sets in Time Series
This work addresses the problem of evaluating motif discovery methods for researchers and practitioners in time series analysis, offering an incremental improvement in benchmarking tools.
The paper tackles the lack of quantitative evaluation in time series motif discovery by introducing PROM, a new metric that overcomes limitations of existing ones, and TSMD-Bench, a challenging benchmark, showing that PROM provides more comprehensive evaluation and enables systematic performance comparisons.
Time Series Motif Discovery (TSMD), which aims at finding recurring patterns in time series, is an important task in numerous application domains, and many methods for this task exist. These methods are usually evaluated qualitatively. A few metrics for quantitative evaluation, where discovered motifs are compared to some ground truth, have been proposed, but they typically make implicit assumptions that limit their applicability. This paper introduces PROM, a broadly applicable metric that overcomes those limitations, and TSMD-Bench, a benchmark for quantitative evaluation of time series motif discovery. Experiments with PROM and TSMD-Bench show that PROM provides a more comprehensive evaluation than existing metrics, that TSMD-Bench is a more challenging benchmark than earlier ones, and that the combination can help understand the relative performance of TSMD methods. More generally, the proposed approach enables large-scale, systematic performance comparisons in this field.