Preventing posterior collapse in variational autoencoders for text generation via decoder regularization
This addresses a specific issue in text generation models, but it is incremental as it builds on existing regularization techniques.
The paper tackles the posterior collapse problem in variational autoencoders for text generation by proposing a decoder regularization method based on fraternal dropout, resulting in improvements across all tested configurations.
Variational autoencoders trained to minimize the reconstruction error are sensitive to the posterior collapse problem, that is the proposal posterior distribution is always equal to the prior. We propose a novel regularization method based on fraternal dropout to prevent posterior collapse. We evaluate our approach using several metrics and observe improvements in all the tested configurations.