Bi-fidelity Evolutionary Multiobjective Search for Adversarially Robust Deep Neural Architectures
This work addresses the challenge of efficiently finding robust architectures for security-sensitive applications, but it is incremental as it builds on existing NAS and adversarial training methods.
The paper tackles the problem of computationally expensive neural architecture search for adversarially robust deep neural networks by proposing a bi-fidelity multiobjective approach, achieving competitive results on datasets like CIFAR-10, CIFAR-100, and SVHN.
Deep neural networks have been found vulnerable to adversarial attacks, thus raising potentially concerns in security-sensitive contexts. To address this problem, recent research has investigated the adversarial robustness of deep neural networks from the architectural point of view. However, searching for architectures of deep neural networks is computationally expensive, particularly when coupled with adversarial training process. To meet the above challenge, this paper proposes a bi-fidelity multiobjective neural architecture search approach. First, we formulate the NAS problem for enhancing adversarial robustness of deep neural networks into a multiobjective optimization problem. Specifically, in addition to a low-fidelity performance predictor as the first objective, we leverage an auxiliary-objective -- the value of which is the output of a surrogate model trained with high-fidelity evaluations. Secondly, we reduce the computational cost by combining three performance estimation methods, i.e., parameter sharing, low-fidelity evaluation, and surrogate-based predictor. The effectiveness of the proposed approach is confirmed by extensive experiments conducted on CIFAR-10, CIFAR-100 and SVHN datasets.