Improve black-box sequential anomaly detector relevancy with limited user feedback
This work addresses the issue for end-users who need more relevant anomaly detection in applications, but it is incremental as it builds on existing detectors without modifying their internal mechanisms.
The paper tackled the problem of improving the relevancy of black-box sequential anomaly detectors to user interests by incorporating limited human feedback, resulting in significant improvements in precision and recall across various datasets.
Anomaly detectors are often designed to catch statistical anomalies. End-users typically do not have interest in all of the detected outliers, but only those relevant to their application. Given an existing black-box sequential anomaly detector, this paper proposes a method to improve its user relevancy using a small number of human feedback. As our first contribution, the method is agnostic to the detector: it only assumes access to its anomaly scores, without requirement on any additional information inside it. Inspired by a fact that anomalies are of different types, our approach identifies these types and utilizes user feedback to assign relevancy to types. This relevancy score, as our second contribution, is used to adjust the subsequent anomaly selection process. Empirical results on synthetic and real-world datasets show that our approach yields significant improvements on precision and recall over a range of anomaly detectors.