Black-box Unsupervised Domain Adaptation with Bi-directional Atkinson-Shiffrin Memory
This work addresses data privacy and flexibility issues in domain adaptation for visual recognition, though it is incremental as it builds on existing black-box UDA methods.
The paper tackles the problem of noisy pseudo labels in black-box unsupervised domain adaptation by proposing BiMem, a bi-directional memorization mechanism that corrects labels on the fly, achieving superior performance across image classification, semantic segmentation, and object detection tasks.
Black-box unsupervised domain adaptation (UDA) learns with source predictions of target data without accessing either source data or source models during training, and it has clear superiority in data privacy and flexibility in target network selection. However, the source predictions of target data are often noisy and training with them is prone to learning collapses. We propose BiMem, a bi-directional memorization mechanism that learns to remember useful and representative information to correct noisy pseudo labels on the fly, leading to robust black-box UDA that can generalize across different visual recognition tasks. BiMem constructs three types of memory, including sensory memory, short-term memory, and long-term memory, which interact in a bi-directional manner for comprehensive and robust memorization of learnt features. It includes a forward memorization flow that identifies and stores useful features and a backward calibration flow that rectifies features' pseudo labels progressively. Extensive experiments show that BiMem achieves superior domain adaptation performance consistently across various visual recognition tasks such as image classification, semantic segmentation and object detection.