Quadratic Upper Bound for Boosting Robustness
This work addresses robustness degradation in fast adversarial training for machine learning models, representing an incremental improvement.
The paper tackles the problem of compromised robustness in fast adversarial training (FAT) by developing a quadratic upper bound (QUB) loss function, which when applied to existing FAT methods yields significant improvement in robustness.
Fast adversarial training (FAT) aims to enhance the robustness of models against adversarial attacks with reduced training time, however, FAT often suffers from compromised robustness due to insufficient exploration of adversarial space. In this paper, we develop a loss function to mitigate the problem of degraded robustness under FAT. Specifically, we derive a quadratic upper bound (QUB) on the adversarial training (AT) loss function and propose to utilize the bound with existing FAT methods. Our experimental results show that applying QUB loss to the existing methods yields significant improvement of robustness. Furthermore, using various metrics, we demonstrate that this improvement is likely to result from the smoothened loss landscape of the resulting model.