Robust Regularization with Adversarial Labelling of Perturbed Samples
This addresses the problem of balancing accuracy and robustness in neural networks for machine learning practitioners, representing an incremental improvement by combining existing regularization principles with adversarial training.
The paper tackles the challenge of improving both generalization and adversarial robustness in neural networks by proposing ALPS, a regularization scheme based on Vicinal Risk Minimization that uses adversarially labeled perturbed samples, achieving state-of-the-art performance on datasets like SVHN, CIFAR-10, CIFAR-100, and Tiny-ImageNet.
Recent researches have suggested that the predictive accuracy of neural network may contend with its adversarial robustness. This presents challenges in designing effective regularization schemes that also provide strong adversarial robustness. Revisiting Vicinal Risk Minimization (VRM) as a unifying regularization principle, we propose Adversarial Labelling of Perturbed Samples (ALPS) as a regularization scheme that aims at improving the generalization ability and adversarial robustness of the trained model. ALPS trains neural networks with synthetic samples formed by perturbing each authentic input sample towards another one along with an adversarially assigned label. The ALPS regularization objective is formulated as a min-max problem, in which the outer problem is minimizing an upper-bound of the VRM loss, and the inner problem is L$_1$-ball constrained adversarial labelling on perturbed sample. The analytic solution to the induced inner maximization problem is elegantly derived, which enables computational efficiency. Experiments on the SVHN, CIFAR-10, CIFAR-100 and Tiny-ImageNet datasets show that the ALPS has a state-of-the-art regularization performance while also serving as an effective adversarial training scheme.