Robust NAS under adversarial training: benchmark, theory, and beyond
This work addresses the problem of ensuring robust architectures in NAS for the machine learning community, providing foundational resources and theoretical insights, though it is incremental in building upon existing NAS and adversarial training methods.
The paper tackles the lack of benchmark evaluations and theoretical guarantees for neural architecture search (NAS) under adversarial training by releasing a comprehensive dataset with clean and robust accuracy metrics for adversarially trained networks and establishing a generalization theory using neural tangent kernel (NTK) tools.
Recent developments in neural architecture search (NAS) emphasize the significance of considering robust architectures against malicious data. However, there is a notable absence of benchmark evaluations and theoretical guarantees for searching these robust architectures, especially when adversarial training is considered. In this work, we aim to address these two challenges, making twofold contributions. First, we release a comprehensive data set that encompasses both clean accuracy and robust accuracy for a vast array of adversarially trained networks from the NAS-Bench-201 search space on image datasets. Then, leveraging the neural tangent kernel (NTK) tool from deep learning theory, we establish a generalization theory for searching architecture in terms of clean accuracy and robust accuracy under multi-objective adversarial training. We firmly believe that our benchmark and theoretical insights will significantly benefit the NAS community through reliable reproducibility, efficient assessment, and theoretical foundation, particularly in the pursuit of robust architectures.