GeoNet: Benchmarking Unsupervised Adaptation across Geographies
This addresses the challenge of fair and inclusive computer vision for models deployed in underrepresented geographies, though it is incremental as it focuses on benchmarking and analysis rather than proposing a new solution.
The paper tackles the problem of geographic robustness in vision models by introducing the GeoNet dataset for benchmarking unsupervised adaptation across geographies, and finds that existing state-of-the-art domain adaptation methods and large-scale pre-training fail to achieve geographic robustness.
In recent years, several efforts have been aimed at improving the robustness of vision models to domains and environments unseen during training. An important practical problem pertains to models deployed in a new geography that is under-represented in the training dataset, posing a direct challenge to fair and inclusive computer vision. In this paper, we study the problem of geographic robustness and make three main contributions. First, we introduce a large-scale dataset GeoNet for geographic adaptation containing benchmarks across diverse tasks like scene recognition (GeoPlaces), image classification (GeoImNet) and universal adaptation (GeoUniDA). Second, we investigate the nature of distribution shifts typical to the problem of geographic adaptation and hypothesize that the major source of domain shifts arise from significant variations in scene context (context shift), object design (design shift) and label distribution (prior shift) across geographies. Third, we conduct an extensive evaluation of several state-of-the-art unsupervised domain adaptation algorithms and architectures on GeoNet, showing that they do not suffice for geographical adaptation, and that large-scale pre-training using large vision models also does not lead to geographic robustness. Our dataset is publicly available at https://tarun005.github.io/GeoNet.