Diversity Boosted Learning for Domain Generalization with Large Number of Domains
This work addresses domain generalization for machine learning models, focusing on improving efficiency and robustness against spurious correlations, but it appears incremental as it builds on existing DG methods like Causal Matching and FISH.
The paper tackles the problem of domain generalization (DG) by addressing the inefficiency of existing methods that require O(n^2) pairwise domain operations and the overlooked issue of spurious correlations from objects, proposing a framework called DOMI that uses Determinantal Point Processes to sample informative domains efficiently. The result is a substantial enhancement in performance on datasets like rotated MNIST, rotated Fashion MNIST, and iwildcam, though specific numbers are not provided in the abstract.
Machine learning algorithms minimizing the average training loss usually suffer from poor generalization performance due to the greedy exploitation of correlations among the training data, which are not stable under distributional shifts. It inspires various works for domain generalization (DG), where a series of methods, such as Causal Matching and FISH, work by pairwise domain operations. They would need $O(n^2)$ pairwise domain operations with $n$ domains, where each one is often highly expensive. Moreover, while a common objective in the DG literature is to learn invariant representations against domain-induced spurious correlations, we highlight the importance of mitigating spurious correlations caused by objects. Based on the observation that diversity helps mitigate spurious correlations, we propose a Diversity boosted twO-level saMplIng framework (DOMI) utilizing Determinantal Point Processes (DPPs) to efficiently sample the most informative ones among large number of domains. We show that DOMI helps train robust models against spurious correlations from both domain-side and object-side, substantially enhancing the performance of the backbone DG algorithms on rotated MNIST, rotated Fashion MNIST, and iwildcam datasets.