Bias and Generalization in Deep Generative Models: An Empirical Study
This study addresses the lack of understanding in inductive bias for deep generative models, which is crucial for high-dimensional density estimation, but it is incremental as it builds on existing models without introducing new methods.
The authors tackled the problem of understanding inductive bias and generalization in deep generative models for images by proposing a framework to systematically investigate when and how these models generate novel attributes and combinations, identifying similarities to human psychology and verifying consistency across models.
In high dimensional settings, density estimation algorithms rely crucially on their inductive bias. Despite recent empirical success, the inductive bias of deep generative models is not well understood. In this paper we propose a framework to systematically investigate bias and generalization in deep generative models of images. Inspired by experimental methods from cognitive psychology, we probe each learning algorithm with carefully designed training datasets to characterize when and how existing models generate novel attributes and their combinations. We identify similarities to human psychology and verify that these patterns are consistent across commonly used models and architectures.