Posterior Collapse of a Linear Latent Variable Model
This addresses a common issue in Bayesian deep learning, providing theoretical insights but is incremental as it focuses on a simplified linear case.
The paper identifies the cause of posterior collapse in a linear latent variable model, showing it results from competition between likelihood and prior regularization, and suggests links to neural and dimensional collapse in deeper architectures.
This work identifies the existence and cause of a type of posterior collapse that frequently occurs in the Bayesian deep learning practice. For a general linear latent variable model that includes linear variational autoencoders as a special case, we precisely identify the nature of posterior collapse to be the competition between the likelihood and the regularization of the mean due to the prior. Our result suggests that posterior collapse may be related to neural collapse and dimensional collapse and could be a subclass of a general problem of learning for deeper architectures.