Learning Correlated Latent Representations with Adaptive Priors
This work addresses the limitation of existing methods in capturing full correlations in data for applications like link prediction and clustering, though it appears incremental as it builds directly on CVAEs.
The authors tackled the problem of learning latent representations that capture all available correlations in data, which previous methods like CVAEs failed to guarantee, by proposing ACVAEs with an adaptive prior and tractable joint variational distribution. Experimental results showed that ACVAEs significantly outperform CVAEs and other benchmarks in tasks like link prediction and hierarchical clustering.
Variational Auto-Encoders (VAEs) have been widely applied for learning compact, low-dimensional latent representations of high-dimensional data. When the correlation structure among data points is available, previous work proposed Correlated Variational Auto-Encoders (CVAEs), which employ a structured mixture model as prior and a structured variational posterior for each mixture component to enforce that the learned latent representations follow the same correlation structure. However, as we demonstrate in this work, such a choice cannot guarantee that CVAEs capture all the correlations. Furthermore, it prevents us from obtaining a tractable joint and marginal variational distribution. To address these issues, we propose Adaptive Correlated Variational Auto-Encoders (ACVAEs), which apply an adaptive prior distribution that can be adjusted during training and can learn a tractable joint variational distribution. Its tractable form also enables further refinement with belief propagation. Experimental results on link prediction and hierarchical clustering show that ACVAEs significantly outperform CVAEs among other benchmarks.