Domain Agnostic Conditional Invariant Predictions for Domain Generalization
This addresses domain generalization in scenarios where domain labels are unavailable, which is common in real-world applications, representing a novel method for a known bottleneck.
The paper tackles domain generalization without requiring domain labels by proposing Discriminant Risk Minimization (DRM) theory and an algorithm using Bayesian inference and a Categorical Discriminant Risk penalty to capture invariant features, showing empirical support on real-world datasets.
Domain generalization aims to develop a model that can perform well on unseen target domains by learning from multiple source domains. However, recent-proposed domain generalization models usually rely on domain labels, which may not be available in many real-world scenarios. To address this challenge, we propose a Discriminant Risk Minimization (DRM) theory and the corresponding algorithm to capture the invariant features without domain labels. In DRM theory, we prove that reducing the discrepancy of prediction distribution between overall source domain and any subset of it can contribute to obtaining invariant features. To apply the DRM theory, we develop an algorithm which is composed of Bayesian inference and a new penalty termed as Categorical Discriminant Risk (CDR). In Bayesian inference, we transform the output of the model into a probability distribution to align with our theoretical assumptions. We adopt sliding update approach to approximate the overall prediction distribution of the model, which enables us to obtain CDR penalty. We also indicate the effectiveness of these components in finding invariant features. We evaluate our algorithm against various domain generalization methods on multiple real-world datasets, providing empirical support for our theory.