Deep Stable Learning for Out-Of-Distribution Generalization
This addresses the challenge of out-of-distribution generalization for deep learning models, particularly in scenarios without domain labels or equal domain capacities, representing an incremental advance in domain generalization.
The paper tackles the problem of deep neural networks failing under distribution shifts by proposing a method to remove feature dependencies via sample weighting, eliminating spurious correlations. It demonstrates effectiveness on benchmarks like PACS, VLCS, MNIST-M, and NICO, outperforming state-of-the-art methods.
Approaches based on deep neural networks have achieved striking performance when testing data and training data share similar distribution, but can significantly fail otherwise. Therefore, eliminating the impact of distribution shifts between training and testing data is crucial for building performance-promising deep models. Conventional methods assume either the known heterogeneity of training data (e.g. domain labels) or the approximately equal capacities of different domains. In this paper, we consider a more challenging case where neither of the above assumptions holds. We propose to address this problem by removing the dependencies between features via learning weights for training samples, which helps deep models get rid of spurious correlations and, in turn, concentrate more on the true connection between discriminative features and labels. Extensive experiments clearly demonstrate the effectiveness of our method on multiple distribution generalization benchmarks compared with state-of-the-art counterparts. Through extensive experiments on distribution generalization benchmarks including PACS, VLCS, MNIST-M, and NICO, we show the effectiveness of our method compared with state-of-the-art counterparts.