NCSAM Noise-Compensated Sharpness-Aware Minimization for Noisy Label Learning
For deep learning practitioners dealing with noisy labels, NCSAM offers a simple optimization-based approach that mitigates memorization without complex label correction or sample selection.
NCSAM proposes a noise-compensated perturbation for Sharpness-Aware Minimization to counteract optimization bias from noisy labels, achieving consistent improvements over SAM baselines and remaining competitive with existing noisy-label learning methods.
Learning from Noisy Labels (LNL) remains a fundamental challenge in deep learning because real-world datasets often contain corrupted annotations. Most existing methods rely on label correction or sample selection mechanisms. In contrast, we study LNL from an optimization perspective by establishing a theoretical connection between label noise and the flatness-seeking behavior of Sharpness-Aware Minimization (SAM). Based on this analysis, we propose Noise-Compensated Sharpness-Aware Minimization (NCSAM), which uses a noise-compensated perturbation to counteract the optimization bias induced by noisy labels. By correcting distorted SAM perturbations, NCSAM mitigates the memorization of noisy labels during training while preserving the simplicity of optimization-based learning. Experiments on synthetic and real-world noisy-label benchmarks show that NCSAM consistently improves over SAM-based optimization baselines and remains competitive with representative noisy-label learning methods.