Regularization Penalty Optimization for Addressing Data Quality Variance in OoD Algorithms
This addresses a specific bottleneck in OoD generalization for machine learning applications, but it is incremental as it builds on existing Lipschitz regularized invariant risk minimization methods.
The paper tackles the problem of Out-of-Distribution (OoD) generalization algorithms being compromised by variance in training data quality, proposing a novel algorithm that alleviates this influence at sample and domain levels, with experiments on regression and classification benchmarks validating its effectiveness.
Due to the poor generalization performance of traditional empirical risk minimization (ERM) in the case of distributional shift, Out-of-Distribution (OoD) generalization algorithms receive increasing attention. However, OoD generalization algorithms overlook the great variance in the quality of training data, which significantly compromises the accuracy of these methods. In this paper, we theoretically reveal the relationship between training data quality and algorithm performance and analyze the optimal regularization scheme for Lipschitz regularized invariant risk minimization. A novel algorithm is proposed based on the theoretical results to alleviate the influence of low-quality data at both the sample level and the domain level. The experiments on both the regression and classification benchmarks validate the effectiveness of our method with statistical significance.