Towards Robust Dataset Learning
This addresses the problem of slow adversarial training for computer vision researchers by enabling faster natural training with robust datasets, though it is incremental as it builds on existing adversarial robustness concepts.
The paper tackles the high computational cost of adversarial training by learning a robust dataset that enables any classifier trained on it to be adversarially robust, achieving effectiveness across MNIST, CIFAR10, and TinyImageNet datasets.
Adversarial training has been actively studied in recent computer vision research to improve the robustness of models. However, due to the huge computational cost of generating adversarial samples, adversarial training methods are often slow. In this paper, we study the problem of learning a robust dataset such that any classifier naturally trained on the dataset is adversarially robust. Such a dataset benefits the downstream tasks as natural training is much faster than adversarial training, and demonstrates that the desired property of robustness is transferable between models and data. In this work, we propose a principled, tri-level optimization to formulate the robust dataset learning problem. We show that, under an abstraction model that characterizes robust vs. non-robust features, the proposed method provably learns a robust dataset. Extensive experiments on MNIST, CIFAR10, and TinyImageNet demostrate the effectiveness of our algorithm with different network initializations and architectures.