Encoder-Powered Generative Adversarial Networks
This work addresses a bottleneck in generative modeling for multi-manifold data, offering a novel method that is incremental in improving GAN architectures.
The paper tackles the problem of learning multi-manifold data structures and abstract features in generative models by introducing EncGAN, which uses an encoder to model manifold structure and invert it for generation, resulting in successful learning on datasets like MNIST, 3D-chair, and UT-Zap50k with disentangled features enabling style transfer.
We present an encoder-powered generative adversarial network (EncGAN) that is able to learn both the multi-manifold structure and the abstract features of data. Unlike the conventional decoder-based GANs, EncGAN uses an encoder to model the manifold structure and invert the encoder to generate data. This unique scheme enables the proposed model to exclude discrete features from the smooth structure modeling and learn multi-manifold data without being hindered by the disconnections. Also, as EncGAN requires a single latent space to carry the information for all the manifolds, it builds abstract features shared among the manifolds in the latent space. For an efficient computation, we formulate EncGAN using a simple regularizer, and mathematically prove its validity. We also experimentally demonstrate that EncGAN successfully learns the multi-manifold structure and the abstract features of MNIST, 3D-chair and UT-Zap50k datasets. Our analysis shows that the learned abstract features are disentangled and make a good style-transfer even when the source data is off the trained distribution.