Robust and Generalizable Visual Representation Learning via Random Convolutions
This work addresses the problem of domain generalization in computer vision for applications requiring robust visual representations, though it is incremental as it builds on existing data augmentation techniques.
The paper tackles the vulnerability of deep neural networks to texture style shifts and small perturbations by using random convolutions as data augmentation to improve robustness and generalizability. The method significantly outperforms state-of-the-art methods on domain generalization benchmarks, such as sketch domains in PACS and ImageNet-Sketch, and is scalable to ImageNet.
While successful for various computer vision tasks, deep neural networks have shown to be vulnerable to texture style shifts and small perturbations to which humans are robust. In this work, we show that the robustness of neural networks can be greatly improved through the use of random convolutions as data augmentation. Random convolutions are approximately shape-preserving and may distort local textures. Intuitively, randomized convolutions create an infinite number of new domains with similar global shapes but random local textures. Therefore, we explore using outputs of multi-scale random convolutions as new images or mixing them with the original images during training. When applying a network trained with our approach to unseen domains, our method consistently improves the performance on domain generalization benchmarks and is scalable to ImageNet. In particular, in the challenging scenario of generalizing to the sketch domain in PACS and to ImageNet-Sketch, our method outperforms state-of-art methods by a large margin. More interestingly, our method can benefit downstream tasks by providing a more robust pretrained visual representation.