Conditional Generative Moment-Matching Networks
This work addresses conditional distribution learning for generative modeling, but it appears incremental as it builds on existing MMD methods with a conditional extension.
The paper tackles the problem of learning conditional distributions by introducing Conditional Generative Moment-Matching Networks (CGMMN), which use a conditional maximum mean discrepancy criterion, and demonstrates competitive performance across tasks like predictive modeling and Bayesian dark knowledge.
Maximum mean discrepancy (MMD) has been successfully applied to learn deep generative models for characterizing a joint distribution of variables via kernel mean embedding. In this paper, we present conditional generative moment- matching networks (CGMMN), which learn a conditional distribution given some input variables based on a conditional maximum mean discrepancy (CMMD) criterion. The learning is performed by stochastic gradient descent with the gradient calculated by back-propagation. We evaluate CGMMN on a wide range of tasks, including predictive modeling, contextual generation, and Bayesian dark knowledge, which distills knowledge from a Bayesian model by learning a relatively small CGMMN student network. Our results demonstrate competitive performance in all the tasks.