Bi-Level Optimization for Single Domain Generalization
This addresses the challenge of robust machine learning for applications where only one labeled source domain is available, but it is incremental as it builds on existing SDG methods.
The paper tackles the problem of Single Domain Generalization (SDG), where models must generalize from a single source domain to unseen target domains without target data during training, by proposing BiSDG, a bi-level optimization framework that decouples task learning from domain modeling, and it achieves new state-of-the-art performance on various SDG benchmarks.
Generalizing from a single labeled source domain to unseen target domains, without access to any target data during training, remains a fundamental challenge in robust machine learning. We address this underexplored setting, known as Single Domain Generalization (SDG), by proposing BiSDG, a bi-level optimization framework that explicitly decouples task learning from domain modeling. BiSDG simulates distribution shifts through surrogate domains constructed via label-preserving transformations of the source data. To capture domain-specific context, we propose a domain prompt encoder that generates lightweight modulation signals to produce augmenting features via feature-wise linear modulation. The learning process is formulated as a bi-level optimization problem: the inner objective optimizes task performance under fixed prompts, while the outer objective maximizes generalization across the surrogate domains by updating the domain prompt encoder. We further develop a practical gradient approximation scheme that enables efficient bi-level training without second-order derivatives. Extensive experiments on various SGD benchmarks demonstrate that BiSDG consistently outperforms prior methods, setting new state-of-the-art performance in the SDG setting.