Improved Anomaly Detection by Using the Attention-Based Isolation Forest
This work addresses anomaly detection, a key problem in data analysis, by introducing an incremental improvement to the Isolation Forest method.
The authors tackled anomaly detection by proposing Attention-Based Isolation Forest (ABIForest), a modification that incorporates an attention mechanism via Nadaraya-Watson regression to assign learnable weights to tree paths, resulting in improved performance as shown in numerical experiments with synthetic and real datasets.
A new modification of Isolation Forest called Attention-Based Isolation Forest (ABIForest) for solving the anomaly detection problem is proposed. It incorporates the attention mechanism in the form of the Nadaraya-Watson regression into the Isolation Forest for improving solution of the anomaly detection problem. The main idea underlying the modification is to assign attention weights to each path of trees with learnable parameters depending on instances and trees themselves. The Huber's contamination model is proposed to be used for defining the attention weights and their parameters. As a result, the attention weights are linearly depend on the learnable attention parameters which are trained by solving the standard linear or quadratic optimization problem. ABIForest can be viewed as the first modification of Isolation Forest, which incorporates the attention mechanism in a simple way without applying gradient-based algorithms. Numerical experiments with synthetic and real datasets illustrate outperforming results of ABIForest. The code of proposed algorithms is available.