A Hybrid Convolutional Variational Autoencoder for Text Generation
This work addresses challenges in training VAEs for text generation, offering incremental improvements for researchers in natural language processing.
The authors tackled the problem of improving Variational Autoencoders for text generation by proposing a hybrid architecture combining convolutional and recurrent components, resulting in faster runtime, better convergence, and improved handling of long sequences.
In this paper we explore the effect of architectural choices on learning a Variational Autoencoder (VAE) for text generation. In contrast to the previously introduced VAE model for text where both the encoder and decoder are RNNs, we propose a novel hybrid architecture that blends fully feed-forward convolutional and deconvolutional components with a recurrent language model. Our architecture exhibits several attractive properties such as faster run time and convergence, ability to better handle long sequences and, more importantly, it helps to avoid some of the major difficulties posed by training VAE models on textual data.