An Efficient Adversarial Attack for Tree Ensembles
This work addresses the challenge of adversarial attacks for machine learning practitioners using tree ensembles, offering a more efficient and effective solution compared to existing methods.
The paper tackles the problem of efficiently generating adversarial examples for tree-based ensembles like GBDTs and random forests, which are non-continuous and lack gradients, by formulating it as a discrete search for leaf tuples and using a greedy algorithm; experimental results show the method is thousands of times faster than previous MILP-based approaches and produces smaller adversarial examples than decision-based black-box attacks.
We study the problem of efficient adversarial attacks on tree based ensembles such as gradient boosting decision trees (GBDTs) and random forests (RFs). Since these models are non-continuous step functions and gradient does not exist, most existing efficient adversarial attacks are not applicable. Although decision-based black-box attacks can be applied, they cannot utilize the special structure of trees. In our work, we transform the attack problem into a discrete search problem specially designed for tree ensembles, where the goal is to find a valid "leaf tuple" that leads to mis-classification while having the shortest distance to the original input. With this formulation, we show that a simple yet effective greedy algorithm can be applied to iteratively optimize the adversarial example by moving the leaf tuple to its neighborhood within hamming distance 1. Experimental results on several large GBDT and RF models with up to hundreds of trees demonstrate that our method can be thousands of times faster than the previous mixed-integer linear programming (MILP) based approach, while also providing smaller (better) adversarial examples than decision-based black-box attacks on general $\ell_p$ ($p=1, 2, \infty$) norm perturbations. Our code is available at https://github.com/chong-z/tree-ensemble-attack.