A Testable Certificate for Constant Collapse in Teacher-Guided VAEs
Provides a testable certificate for a specific failure mode (constant collapse) in VAEs, enabling quantitative verification of latent code usage, but is narrow in scope and does not address all forms of posterior collapse.
The authors derive an exact threshold for input-independent constant collapse in teacher-guided VAEs, showing that a latent-only witness achieving alignment loss below the teacher mutual information cannot be constant. Experiments on CIFAR-100 and Tiny-ImageNet-200 demonstrate that full training stays certified, removing alignment causes collapse, and restarting from collapsed checkpoints restores the certificate, while standard baselines fail the certificate.
Posterior collapse in variational autoencoders is often diagnosed by its symptoms: a small KL term, a strong decoder, or weak use of the latent code. These signals are useful, but they do not define a collapse boundary. We study a concrete failure mode, input-independent constant collapse, and show that this case admits an exact threshold. For any fixed nonconstant teacher distribution \(T(\cdot\mid x)\), the best constant student is the dataset-average teacher distribution, and its alignment cost is the teacher mutual information \(I_T(X;T)\). Therefore, if a strictly latent-only raw witness achieves alignment loss below this value, with a safety margin, the witness cannot be constant in the input. This identity turns a qualitative failure mode into a measurable one. In CIFAR-100 experiments with per-seed teacher search, full training stays on the certified side of the boundary, removing alignment drives the raw witness into the constant-student regime, and restarting from a collapsed checkpoint with alignment enabled restores the certificate. Tiny-ImageNet-200 fixed-target runs show the same prevention--collapse--rescue pattern across three independently searched teachers. Standard VAE-style baselines, including methods that preserve reconstruction quality or post-hoc predictability, remain negative under the raw certificate. The guarantee is intentionally narrow: it certifies that the matched nonconstant teacher-relative variation passes through the latent pathway, rather than claiming that all forms of posterior collapse have been ruled out.