Towards Principled Continual Anomaly Detection: A Systematic Framework and Benchmark Scenarios
For researchers in continual anomaly detection, this work provides a principled methodology to create meaningful benchmarks, addressing a key bottleneck in evaluating CAD methods.
The paper introduces a systematic framework for designing reproducible benchmark scenarios for continual anomaly detection (CAD) in tabular domains, addressing issues like arbitrary task splits that obscure genuine continual learning. The framework discovers, filters, and orders tasks, producing five benchmark scenarios from three cybersecurity datasets.
Continual anomaly detection (CAD) studies how models can adapt to evolving data distributions while retaining performance on previously observed regimes. CAD benchmarks, however, depend critically on how tasks are defined, filtered, ordered, and validated. In tabular domains, task boundaries are rarely given, and arbitrary splits can create unlearnable, redundant, or overly transferable tasks that obscure genuine continual-learning behavior. To this end, we introduce a systematic framework for reproducible benchmark scenario design from existing tabular anomaly-detection datasets. The framework discovers candidate tasks, filters unsuitable tasks, and derives principled orderings that expose diverse dynamics. The framework allows us to deliver five benchmark-ready scenarios from three large-scale cybersecurity anomaly detection datasets, yielding both single-dataset and multi-dataset CAD settings.