Learning Gradient-based Mixup towards Flatter Minima for Domain Generalization
This addresses domain generalization for machine learning models to improve robustness to unseen data distributions, representing an incremental advance.
The paper tackles overfitting in domain generalization by proposing Flatness-aware Gradient-based Mixup (FGMix), which uses gradient-based similarity to weight instances for mixup, achieving superior performance on the DomainBed benchmark.
To address the distribution shifts between training and test data, domain generalization (DG) leverages multiple source domains to learn a model that generalizes well to unseen domains. However, existing DG methods generally suffer from overfitting to the source domains, partly due to the limited coverage of the expected region in feature space. Motivated by this, we propose to perform mixup with data interpolation and extrapolation to cover the potential unseen regions. To prevent the detrimental effects of unconstrained extrapolation, we carefully design a policy to generate the instance weights, named Flatness-aware Gradient-based Mixup (FGMix). The policy employs a gradient-based similarity to assign greater weights to instances that carry more invariant information, and learns the similarity function towards flatter minima for better generalization. On the DomainBed benchmark, we validate the efficacy of various designs of FGMix and demonstrate its superiority over other DG algorithms.