Posterior Collapse as a Phase Transition in Variational Autoencoders
This provides new insights into the trainability and representational capacity of deep generative models, addressing a fundamental issue for researchers in machine learning and statistical physics.
The paper tackled the problem of posterior collapse in variational autoencoders by analyzing it as a phase transition governed by data structure and hyper-parameters, identifying a critical threshold that separates meaningful latent inference from collapse, with validation on synthetic and real-world datasets.
We investigate the phenomenon of posterior collapse in variational autoencoders (VAEs) from the perspective of statistical physics, and reveal that it constitutes a phase transition governed jointly by data structure and model hyper-parameters. By analyzing the stability of the trivial solution associated with posterior collapse, we identify a critical hyper-parameter threshold. This critical boundary, separating meaningful latent inference from collapse, is characterized by a discontinuity in the KL divergence between the approximate posterior and the prior distribution. We validate this critical behavior on both synthetic and real-world datasets, confirming the existence of a phase transition. Our results demonstrate that posterior collapse is not merely an optimization failure, but rather an emerging phase transition arising from the interplay between data structure and variational constraints. This perspective offers new insights into the trainability and representational capacity of deep generative models.