CLID-MU: Cross-Layer Information Divergence Based Meta Update Strategy for Learning with Noisy Labels
This addresses a key bottleneck in learning with noisy labels by eliminating the need for clean meta-data, which is often unavailable in practice, making it a novel method for a known bottleneck.
The paper tackles the problem of meta-learning for noisy label scenarios without requiring a clean labeled dataset, by proposing CLID-MU, which leverages cross-layer information divergence to guide training, and it outperforms state-of-the-art methods on benchmark datasets with synthetic and real-world noise.
Learning with noisy labels (LNL) is essential for training deep neural networks with imperfect data. Meta-learning approaches have achieved success by using a clean unbiased labeled set to train a robust model. However, this approach heavily depends on the availability of a clean labeled meta-dataset, which is difficult to obtain in practice. In this work, we thus tackle the challenge of meta-learning for noisy label scenarios without relying on a clean labeled dataset. Our approach leverages the data itself while bypassing the need for labels. Building on the insight that clean samples effectively preserve the consistency of related data structures across the last hidden and the final layer, whereas noisy samples disrupt this consistency, we design the Cross-layer Information Divergence-based Meta Update Strategy (CLID-MU). CLID-MU leverages the alignment of data structures across these diverse feature spaces to evaluate model performance and use this alignment to guide training. Experiments on benchmark datasets with varying amounts of labels under both synthetic and real-world noise demonstrate that CLID-MU outperforms state-of-the-art methods. The code is released at https://github.com/ruofanhu/CLID-MU.