A robust assessment for invariant representations
This work addresses a gap in directly assessing invariant performance for IRM-based methods, which is incremental as it focuses on evaluation rather than new learning techniques.
The paper tackles the problem of evaluating how well invariant representations from invariant risk minimization (IRM) methods maintain performance under varying conditions, proposing a novel assessment method based on likelihood ratios and validating it with theory and numerical studies.
The performance of machine learning models can be impacted by changes in data over time. A promising approach to address this challenge is invariant learning, with a particular focus on a method known as invariant risk minimization (IRM). This technique aims to identify a stable data representation that remains effective with out-of-distribution (OOD) data. While numerous studies have developed IRM-based methods adaptive to data augmentation scenarios, there has been limited attention on directly assessing how well these representations preserve their invariant performance under varying conditions. In our paper, we propose a novel method to evaluate invariant performance, specifically tailored for IRM-based methods. We establish a bridge between the conditional expectation of an invariant predictor across different environments through the likelihood ratio. Our proposed criterion offers a robust basis for evaluating invariant performance. We validate our approach with theoretical support and demonstrate its effectiveness through extensive numerical studies.These experiments illustrate how our method can assess the invariant performance of various representation techniques.