A Simple Generative Network
This offers a simpler alternative to complex generative models for researchers and practitioners, though it appears incremental as it builds on existing optimization goals.
The paper tackles the problem of generating samples from complex probability distributions by proposing a simple generative network (SGN) using a single feed-forward neural network and Kullback-Leibler divergence optimization, achieving results visually and quantitatively competitive with state-of-the-art methods like GANs and VAEs.
Generative neural networks are able to mimic intricate probability distributions such as those of handwritten text, natural images, etc. Since their inception several models were proposed. The most successful of these were based on adversarial (GAN), auto-encoding (VAE) and maximum mean discrepancy (MMD) relatively complex architectures and schemes. Surprisingly, a very simple architecture (a single feed-forward neural network) in conjunction with an obvious optimization goal (Kullback_Leibler divergence) was apparently overlooked. This paper demonstrates that such a model (denoted SGN for its simplicity) is able to generate samples visually and quantitatively competitive as compared with the fore-mentioned state of the art methods.