Competitive Training of Mixtures of Independent Deep Generative Models
This work addresses the challenge of unsupervised learning for data with multiple independent sources, offering a method that could enhance generative modeling in domains like image or text synthesis, though it appears incremental as it builds on clustering and adversarial approaches.
The paper tackles the problem of disentangling independent generative mechanisms in data by proposing a competitive training procedure for mixtures of deep generative models, which results in simpler models and improved sample quality.
A common assumption in causal modeling posits that the data is generated by a set of independent mechanisms, and algorithms should aim to recover this structure. Standard unsupervised learning, however, is often concerned with training a single model to capture the overall distribution or aspects thereof. Inspired by clustering approaches, we consider mixtures of implicit generative models that ``disentangle'' the independent generative mechanisms underlying the data. Relying on an additional set of discriminators, we propose a competitive training procedure in which the models only need to capture the portion of the data distribution from which they can produce realistic samples. As a by-product, each model is simpler and faster to train. We empirically show that our approach splits the training distribution in a sensible way and increases the quality of the generated samples.