Ignition Phase : Standard Training for Fast Adversarial Robustness
This addresses the problem of high computational expense in adversarial training for machine learning practitioners, though it is incremental as it builds on existing methods.
The paper tackles the inefficiency of adversarial training by proposing Adversarial Evolution Training (AET), which adds an initial standard training phase to improve feature representations, resulting in comparable or superior robustness achieved more rapidly with 8-25% lower training costs and better clean accuracy.
Adversarial Training (AT) is a cornerstone defense, but many variants overlook foundational feature representations by primarily focusing on stronger attack generation. We introduce Adversarial Evolution Training (AET), a simple yet powerful framework that strategically prepends an Empirical Risk Minimization (ERM) phase to conventional AT. We hypothesize this initial ERM phase cultivates a favorable feature manifold, enabling more efficient and effective robustness acquisition. Empirically, AET achieves comparable or superior robustness more rapidly, improves clean accuracy, and cuts training costs by 8-25\%. Its effectiveness is shown across multiple datasets, architectures, and when augmenting established AT methods. Our findings underscore the impact of feature pre-conditioning via standard training for developing more efficient, principled robust defenses. Code is available in the supplementary material.