LGJan 15, 2013
How good is the Electricity benchmark for evaluating concept drift adaptation
arXiv:1301.3524v155 citations
Originality Synthesis-oriented
AI Analysis
This is an incremental critique for researchers in concept drift adaptation, highlighting a potential flaw in benchmark evaluation methods.
The paper identifies a problem with evaluating adaptive classifiers on autocorrelated data, where random change alarms can artificially inflate accuracy metrics, making it difficult to assess adaptation effectiveness.
In this correspondence, we will point out a problem with testing adaptive classifiers on autocorrelated data. In such a case random change alarms may boost the accuracy figures. Hence, we cannot be sure if the adaptation is working well.