Fine-grained Anomaly Detection in Sequential Data via Counterfactual Explanations
This addresses the need for interpretable anomaly detection in sequential data, such as system logs, but is incremental as it builds on existing methods like Deep SVDD.
The paper tackles the problem of identifying anomalous entries within sequences, which is challenging due to lack of entry-level information, and proposes CFDet, a framework that uses counterfactual explanations to achieve fine-grained detection, with experimental results showing it can correctly detect anomalous entries on three datasets.
Anomaly detection in sequential data has been studied for a long time because of its potential in various applications, such as detecting abnormal system behaviors from log data. Although many approaches can achieve good performance on anomalous sequence detection, how to identify the anomalous entries in sequences is still challenging due to a lack of information at the entry-level. In this work, we propose a novel framework called CFDet for fine-grained anomalous entry detection. CFDet leverages the idea of interpretable machine learning. Given a sequence that is detected as anomalous, we can consider anomalous entry detection as an interpretable machine learning task because identifying anomalous entries in the sequence is to provide an interpretation to the detection result. We make use of the deep support vector data description (Deep SVDD) approach to detect anomalous sequences and propose a novel counterfactual interpretation-based approach to identify anomalous entries in the sequences. Experimental results on three datasets show that CFDet can correctly detect anomalous entries.