Moderately Distributional Exploration for Domain Generalization
This work addresses domain generalization for machine learning models that need to perform well on unseen target domains, but it is incremental as it builds on existing distributionally robust optimization approaches.
The paper tackles the problem of domain generalization by addressing the issue of overly large uncertainty sets in distributionally robust optimization, which can lead to low-confidence predictions due to semantically different factors. The proposed MODE method performs distribution exploration in a subset that shares semantic factors with training domains, achieving competitive performance compared to state-of-the-art baselines.
Domain generalization (DG) aims to tackle the distribution shift between training domains and unknown target domains. Generating new domains is one of the most effective approaches, yet its performance gain depends on the distribution discrepancy between the generated and target domains. Distributionally robust optimization is promising to tackle distribution discrepancy by exploring domains in an uncertainty set. However, the uncertainty set may be overwhelmingly large, leading to low-confidence prediction in DG. It is because a large uncertainty set could introduce domains containing semantically different factors from training domains. To address this issue, we propose to perform a $\textbf{mo}$derately $\textbf{d}$istributional $\textbf{e}$xploration (MODE) for domain generalization. Specifically, MODE performs distribution exploration in an uncertainty $\textit{subset}$ that shares the same semantic factors with the training domains. We show that MODE can endow models with provable generalization performance on unknown target domains. The experimental results show that MODE achieves competitive performance compared to state-of-the-art baselines.