Composite Adversarial Attacks
This work provides a more robust and efficient method for evaluating the adversarial robustness of machine learning models, which is crucial for developers and researchers in ML security.
This paper introduces Composite Adversarial Attack (CAA), an automated method for combining 32 base attack algorithms and their hyperparameters to find the strongest attack policy. CAA outperforms 10 top attackers across 11 defenses, achieving new state-of-the-art for l-infinity, l2, and unrestricted adversarial attacks, while being 6 times faster than AutoAttack.
Adversarial attack is a technique for deceiving Machine Learning (ML) models, which provides a way to evaluate the adversarial robustness. In practice, attack algorithms are artificially selected and tuned by human experts to break a ML system. However, manual selection of attackers tends to be sub-optimal, leading to a mistakenly assessment of model security. In this paper, a new procedure called Composite Adversarial Attack (CAA) is proposed for automatically searching the best combination of attack algorithms and their hyper-parameters from a candidate pool of \textbf{32 base attackers}. We design a search space where attack policy is represented as an attacking sequence, i.e., the output of the previous attacker is used as the initialization input for successors. Multi-objective NSGA-II genetic algorithm is adopted for finding the strongest attack policy with minimum complexity. The experimental result shows CAA beats 10 top attackers on 11 diverse defenses with less elapsed time (\textbf{6 $\times$ faster than AutoAttack}), and achieves the new state-of-the-art on $l_{\infty}$, $l_{2}$ and unrestricted adversarial attacks.