Invariance-inducing regularization using worst-case transformations suffices to boost accuracy and spatial robustness
This addresses the challenge of enhancing model robustness to spatial transformations for computer vision applications, showing incremental improvements over existing methods.
The paper tackles the problem of improving predictive accuracy and spatial robustness against worst-case transformations by using invariance-inducing regularization, achieving a 20% reduction in relative error on CIFAR10 without added computational cost and outperforming handcrafted spatial-equivariant networks.
This work provides theoretical and empirical evidence that invariance-inducing regularizers can increase predictive accuracy for worst-case spatial transformations (spatial robustness). Evaluated on these adversarially transformed examples, we demonstrate that adding regularization on top of standard or adversarial training reduces the relative error by 20% for CIFAR10 without increasing the computational cost. This outperforms handcrafted networks that were explicitly designed to be spatial-equivariant. Furthermore, we observe for SVHN, known to have inherent variance in orientation, that robust training also improves standard accuracy on the test set. We prove that this no-trade-off phenomenon holds for adversarial examples from transformation groups in the infinite data limit.