Accelerating Certified Robustness Training via Knowledge Transfer
This addresses the problem of slow and costly certified training for AI security, enabling more efficient deployment and updates, though it is incremental as it builds on existing methods.
The paper tackles the computational expense of certified robustness training for deep neural networks by proposing a knowledge transfer framework called CRT, which accelerates training by 8x on CIFAR-10 while maintaining comparable robustness to state-of-the-art methods.
Training deep neural network classifiers that are certifiably robust against adversarial attacks is critical to ensuring the security and reliability of AI-controlled systems. Although numerous state-of-the-art certified training methods have been developed, they are computationally expensive and scale poorly with respect to both dataset and network complexity. Widespread usage of certified training is further hindered by the fact that periodic retraining is necessary to incorporate new data and network improvements. In this paper, we propose Certified Robustness Transfer (CRT), a general-purpose framework for reducing the computational overhead of any certifiably robust training method through knowledge transfer. Given a robust teacher, our framework uses a novel training loss to transfer the teacher's robustness to the student. We provide theoretical and empirical validation of CRT. Our experiments on CIFAR-10 show that CRT speeds up certified robustness training by $8 \times$ on average across three different architecture generations while achieving comparable robustness to state-of-the-art methods. We also show that CRT can scale to large-scale datasets like ImageNet.