Learning Representations Robust to Group Shifts and Adversarial Examples
This work addresses robustness issues in deep learning for image tasks, offering an incremental improvement by integrating existing techniques.
The paper tackled the problem of deep neural networks being vulnerable to input perturbations and uncertain distributions by combining adversarial training with group distribution robust optimization. The proposed method achieved superior robust metrics on three image benchmark datasets with minimal sacrifice to standard performance.
Despite the high performance achieved by deep neural networks on various tasks, extensive studies have demonstrated that small tweaks in the input could fail the model predictions. This issue of deep neural networks has led to a number of methods to improve model robustness, including adversarial training and distributionally robust optimization. Though both of these two methods are geared towards learning robust models, they have essentially different motivations: adversarial training attempts to train deep neural networks against perturbations, while distributional robust optimization aims at improving model performance on the most difficult "uncertain distributions". In this work, we propose an algorithm that combines adversarial training and group distribution robust optimization to improve robust representation learning. Experiments on three image benchmark datasets illustrate that the proposed method achieves superior results on robust metrics without sacrificing much of the standard measures.