CR-VAE: Contrastive Regularization on Variational Autoencoders for Preventing Posterior Collapse
This addresses a key limitation in VAEs for researchers and practitioners in machine learning, offering an incremental improvement over existing methods.
The paper tackled the problem of posterior collapse in Variational Autoencoders (VAEs), where latent representations become independent of inputs, by proposing CR-VAE with a contrastive regularization objective to maximize mutual information between similar inputs, and demonstrated that it outperforms state-of-the-art approaches in preventing this issue.
The Variational Autoencoder (VAE) is known to suffer from the phenomenon of \textit{posterior collapse}, where the latent representations generated by the model become independent of the inputs. This leads to degenerated representations of the input, which is attributed to the limitations of the VAE's objective function. In this work, we propose a novel solution to this issue, the Contrastive Regularization for Variational Autoencoders (CR-VAE). The core of our approach is to augment the original VAE with a contrastive objective that maximizes the mutual information between the representations of similar visual inputs. This strategy ensures that the information flow between the input and its latent representation is maximized, effectively avoiding posterior collapse. We evaluate our method on a series of visual datasets and demonstrate, that CR-VAE outperforms state-of-the-art approaches in preventing posterior collapse.