Geometric Gradient Rectification for Safe Open-Set Semi-Supervised Learning
For researchers in semi-supervised learning, this work addresses the gradient conflict problem in open-set scenarios, offering a novel gradient-level control approach that improves upon existing sample-selection methods.
The paper tackles the trade-off in open-set semi-supervised learning between filtering out-of-distribution outliers and utilizing unlabeled data, which can cause gradient conflicts. The proposed Geometric Gradient Rectification (GGR) method improves representative baselines on CIFAR and ImageNet benchmarks, enhancing both closed-set generalization and open-set robustness.
Open-set semi-supervised learning aims to leverage unlabeled data that may contain out-of-distribution outliers while maintaining performance on in-distribution classes. Existing methods mainly follow two paradigms: filtering suspicious samples or incorporating unlabeled objectives with soft weighting. We argue that both face a common trade-off: aggressive filtering can discard informative but hard ID samples, whereas utilization can introduce auxiliary gradients that conflict with supervised learning when pseudo labels are wrong. We therefore shift the focus from sample selection to gradient-level control. We propose \textit{Geometric Gradient Rectification} (GGR), a plug-in framework that uses the supervised gradient as an anchor and projects conflicting auxiliary gradients onto an admissible region in gradient space. This makes the applied auxiliary update first-order non-opposing within the rectified coordinate block while preserving orthogonal components that may still carry useful representation signals. We further extend GGR with subspace-aware rectification to stabilize the anchor under noisy mini-batch gradients. Experiments on CIFAR and ImageNet benchmarks show that GGR improves representative OSSL baselines in most settings and yields gains in both closed-set generalization and open-set robustness. Code will be available at https://github.com/JiaheChen2002/GGR.