RegMix: Adversarial Mutual and Generalization Regularization for Enhancing DNN Robustness
This work addresses the problem of enhancing deep neural network robustness against adversarial attacks for security-critical applications, representing an incremental improvement over existing regularization methods.
The paper tackles the problem of limited robustness in adversarial training due to overly uniform optimization from MSE regularization, by proposing two novel regularization strategies: weighted adversarial mutual regularization and adversarial generalization regularization. The result is a significant improvement in adversarial robustness compared to existing regularization-based approaches, as demonstrated through extensive experiments.
Adversarial training is the most effective defense against adversarial attacks. The effectiveness of the adversarial attacks has been on the design of its loss function and regularization term. The most widely used loss function in adversarial training is cross-entropy and mean squared error (MSE) as its regularization objective. However, MSE enforces overly uniform optimization between two output distributions during training, which limits its robustness in adversarial training scenarios. To address this issue, we revisit the idea of mutual learning (originally designed for knowledge distillation) and propose two novel regularization strategies tailored for adversarial training: (i) weighted adversarial mutual regularization and (ii) adversarial generalization regularization. In the former, we formulate a decomposed adversarial mutual Kullback-Leibler divergence (KL-divergence) loss, which allows flexible control over the optimization process by assigning unequal weights to the main and auxiliary objectives. In the latter, we introduce an additional clean target distribution into the adversarial training objective, improving generalization and enhancing model robustness. Extensive experiments demonstrate that our proposed methods significantly improve adversarial robustness compared to existing regularization-based approaches.