DMT: Dynamic Mutual Training for Semi-Supervised Learning
This addresses the challenge of improving semi-supervised learning accuracy for tasks like image classification and segmentation, though it is incremental as it builds on existing mutual training ideas.
The paper tackles the problem of unreliable pseudo labels in semi-supervised learning by proposing Dynamic Mutual Training (DMT), which uses inter-model disagreement to dynamically re-weight loss and filter errors, achieving state-of-the-art performance in image classification and semantic segmentation.
Recent semi-supervised learning methods use pseudo supervision as core idea, especially self-training methods that generate pseudo labels. However, pseudo labels are unreliable. Self-training methods usually rely on single model prediction confidence to filter low-confidence pseudo labels, thus remaining high-confidence errors and wasting many low-confidence correct labels. In this paper, we point out it is difficult for a model to counter its own errors. Instead, leveraging inter-model disagreement between different models is a key to locate pseudo label errors. With this new viewpoint, we propose mutual training between two different models by a dynamically re-weighted loss function, called Dynamic Mutual Training (DMT). We quantify inter-model disagreement by comparing predictions from two different models to dynamically re-weight loss in training, where a larger disagreement indicates a possible error and corresponds to a lower loss value. Extensive experiments show that DMT achieves state-of-the-art performance in both image classification and semantic segmentation. Our codes are released at https://github.com/voldemortX/DST-CBC .