Fix-A-Step: Semi-supervised Learning from Uncurated Unlabeled Data
This addresses the challenge of applying SSL in real-world scenarios like medical imaging where unlabeled data is often uncurated, offering a practical solution for improved model accuracy without complex out-of-distribution detection.
The paper tackles the problem of semi-supervised learning (SSL) failing with uncurated unlabeled data, which differs from labeled sets in classes or features, by introducing Fix-A-Step, a simpler procedure that uses all uncurated images as potentially helpful augmentations and modifies gradient updates to prevent accuracy loss. It improves accuracy on CIFAR benchmarks across methods and class mismatches, and on a medical SSL benchmark with 353,500 uncurated ultrasound images, delivering gains that generalize across hospitals.
Semi-supervised learning (SSL) promises improved accuracy compared to training classifiers on small labeled datasets by also training on many unlabeled images. In real applications like medical imaging, unlabeled data will be collected for expediency and thus uncurated: possibly different from the labeled set in classes or features. Unfortunately, modern deep SSL often makes accuracy worse when given uncurated unlabeled data. Recent complex remedies try to detect out-of-distribution unlabeled images and then discard or downweight them. Instead, we introduce Fix-A-Step, a simpler procedure that views all uncurated unlabeled images as potentially helpful. Our first insight is that even uncurated images can yield useful augmentations of labeled data. Second, we modify gradient descent updates to prevent optimizing a multi-task SSL loss from hurting labeled-set accuracy. Fix-A-Step can repair many common deep SSL methods, improving accuracy on CIFAR benchmarks across all tested methods and levels of artificial class mismatch. On a new medical SSL benchmark called Heart2Heart, Fix-A-Step can learn from 353,500 truly uncurated ultrasound images to deliver gains that generalize across hospitals.