3D Common Corruptions and Data Augmentation
This addresses robustness issues in computer vision models for real-world applications, though it is incremental by building on existing corruption methods.
The authors tackled the problem of evaluating and improving model robustness by introducing 3D-aware image transformations that incorporate scene geometry and semantic corruptions, showing these expose vulnerabilities in existing models and enhance robustness when used as data augmentation.
We introduce a set of image transformations that can be used as corruptions to evaluate the robustness of models as well as data augmentation mechanisms for training neural networks. The primary distinction of the proposed transformations is that, unlike existing approaches such as Common Corruptions, the geometry of the scene is incorporated in the transformations -- thus leading to corruptions that are more likely to occur in the real world. We also introduce a set of semantic corruptions (e.g. natural object occlusions). We show these transformations are `efficient' (can be computed on-the-fly), `extendable' (can be applied on most image datasets), expose vulnerability of existing models, and can effectively make models more robust when employed as `3D data augmentation' mechanisms. The evaluations on several tasks and datasets suggest incorporating 3D information into benchmarking and training opens up a promising direction for robustness research.