Beyond the Class Subspace: Teacher-Guided Training for Reliable Out-of-Distribution Detection in Single-Domain Models
This addresses the reliability of OOD detection for practical systems trained on single-domain data, which is an incremental improvement over multi-domain methods.
The paper tackled the problem of out-of-distribution (OOD) detection in single-domain models, showing that supervised training causes Domain-Sensitivity Collapse, which reduces sensitivity to domain shifts, and introduced Teacher-Guided Training (TGT) to improve OOD detection, achieving reductions in far-OOD FPR@95 by up to 12.87 percentage points across benchmarks.
Out-of-distribution (OOD) detection methods perform well on multi-domain benchmarks, yet many practical systems are trained on single-domain data. We show that this regime induces a geometric failure mode, Domain-Sensitivity Collapse (DSC): supervised training compresses features into a low-rank class subspace and suppresses directions that carry domain-shift signal. We provide theory showing that, under DSC, distance- and logit-based OOD scores lose sensitivity to domain shift. We then introduce Teacher-Guided Training (TGT), which distills class-suppressed residual structure from a frozen multi-domain teacher (DINOv2) into the student during training. The teacher and auxiliary head are discarded after training, adding no inference overhead. Across eight single-domain benchmarks, TGT yields large far-OOD FPR@95 reductions for distance-based scorers: MDS improves by 11.61 pp, ViM by 10.78 pp, and kNN by 12.87 pp (ResNet-50 average), while maintaining or slightly improving in-domain OOD and classification accuracy.