Text Generation with Deep Variational GAN
This addresses mode-collapse in text generation, an incremental improvement for applications requiring diverse outputs.
The paper tackles the problem of mode-collapsing in deep generative models for sequence generation by proposing a GAN-based framework that modifies the objective to maximize a variational lower-bound of log-likelihood and minimize Jensen-Shannon divergence, resulting in realistic text generation with high diversity.
Generating realistic sequences is a central task in many machine learning applications. There has been considerable recent progress on building deep generative models for sequence generation tasks. However, the issue of mode-collapsing remains a main issue for the current models. In this paper we propose a GAN-based generic framework to address the problem of mode-collapse in a principled approach. We change the standard GAN objective to maximize a variational lower-bound of the log-likelihood while minimizing the Jensen-Shanon divergence between data and model distributions. We experiment our model with text generation task and show that it can generate realistic text with high diversity.