Invariant Learning with Annotation-free Environments
This addresses domain generalization for machine learning applications by reducing annotation dependency, though it appears incremental as it builds on existing invariant learning frameworks.
The paper tackles the problem of domain generalization by proposing an invariant learning method that infers environments without needing pre-partitioned annotations, using properties from an ERM model's representation space. It shows preliminary effectiveness on the ColoredMNIST benchmark, achieving performance comparable to methods requiring explicit labels and on par with an annotation-free method with strong restrictions.
Invariant learning is a promising approach to improve domain generalization compared to Empirical Risk Minimization (ERM). However, most invariant learning methods rely on the assumption that training examples are pre-partitioned into different known environments. We instead infer environments without the need for additional annotations, motivated by observations of the properties within the representation space of a trained ERM model. We show the preliminary effectiveness of our approach on the ColoredMNIST benchmark, achieving performance comparable to methods requiring explicit environment labels and on par with an annotation-free method that poses strong restrictions on the ERM reference model.