Domain Adversarial Neural Networks for Domain Generalization: When It Works and How to Improve
This paper addresses the theoretical and practical limitations of DANN for researchers and practitioners working on domain generalization, offering an incremental improvement.
This paper investigates the theoretical validity of Domain Adversarial Neural Networks (DANN) for domain generalization, finding that its application is not straightforward. The authors propose an algorithmic extension to DANN for domain generalization, which is validated through experimentation.
Theoretically, domain adaptation is a well-researched problem. Further, this theory has been well-used in practice. In particular, we note the bound on target error given by Ben-David et al. (2010) and the well-known domain-aligning algorithm based on this work using Domain Adversarial Neural Networks (DANN) presented by Ganin and Lempitsky (2015). Recently, multiple variants of DANN have been proposed for the related problem of domain generalization, but without much discussion of the original motivating bound. In this paper, we investigate the validity of DANN in domain generalization from this perspective. We investigate conditions under which application of DANN makes sense and further consider DANN as a dynamic process during training. Our investigation suggests that the application of DANN to domain generalization may not be as straightforward as it seems. To address this, we design an algorithmic extension to DANN in the domain generalization case. Our experimentation validates both theory and algorithm.