Fast Adversarial Training against Sparse Attacks Requires Loss Smoothing
This addresses a specific challenge in adversarial robustness for sparse attacks, offering an incremental improvement over existing methods.
The paper tackles the problem of catastrophic overfitting in fast adversarial training against sparse (l0-norm bounded) attacks, showing that it is caused by sub-optimal perturbation locations and a craggy loss landscape. The proposed Fast-LS-l0 method uses loss smoothing to overcome this, achieving state-of-the-art performance and narrowing the gap between 1-step and multi-step training.
This paper studies fast adversarial training against sparse adversarial perturbations bounded by $l_0$ norm. We demonstrate the challenges of employing $1$-step attacks on $l_0$ bounded perturbations for fast adversarial training, including degraded performance and the occurrence of catastrophic overfitting (CO). We highlight that CO in $l_0$ adversarial training is caused by sub-optimal perturbation locations of $1$-step attack. Theoretical and empirical analyses reveal that the loss landscape of $l_0$ adversarial training is more craggy compared to its $l_\infty$, $l_2$ and $l_1$ counterparts. Moreover, we corroborate that the craggy loss landscape can aggravate CO. To address these issues, we propose Fast-LS-$l_0$ that incorporates soft labels and the trade-off loss function to smooth the adversarial loss landscape. Extensive experiments demonstrate our method can overcome the challenge of catastrophic overfitting, achieve state-of-the-art performance, and narrow down the performance gap between $1$-step and multi-step adversarial training against sparse attacks.