Contrastive View Design Strategies to Enhance Robustness to Domain Shifts in Downstream Object Detection
This work addresses robustness issues for object detection systems in real-world applications, but it is incremental as it builds on existing contrastive learning methods.
The paper tackled the problem of robustness in contrastively pretrained object detectors under domain shifts by proposing view design strategies, resulting in improved performance on abstract, weather, and context shifts with specific augmentations like cropping changes and shortcut-reducing techniques.
Contrastive learning has emerged as a competitive pretraining method for object detection. Despite this progress, there has been minimal investigation into the robustness of contrastively pretrained detectors when faced with domain shifts. To address this gap, we conduct an empirical study of contrastive learning and out-of-domain object detection, studying how contrastive view design affects robustness. In particular, we perform a case study of the detection-focused pretext task Instance Localization (InsLoc) and propose strategies to augment views and enhance robustness in appearance-shifted and context-shifted scenarios. Amongst these strategies, we propose changes to cropping such as altering the percentage used, adding IoU constraints, and integrating saliency based object priors. We also explore the addition of shortcut-reducing augmentations such as Poisson blending, texture flattening, and elastic deformation. We benchmark these strategies on abstract, weather, and context domain shifts and illustrate robust ways to combine them, in both pretraining on single-object and multi-object image datasets. Overall, our results and insights show how to ensure robustness through the choice of views in contrastive learning.