Latent-Compressed Variational Autoencoder for Video Diffusion Models
For researchers working on video diffusion models, this work offers a practical solution to improve generative performance without sacrificing reconstruction fidelity.
The paper addresses the trade-off between video reconstruction quality and latent diffusion model performance, proposing a latent compression method that removes high-frequency components instead of reducing channels. The method achieves superior reconstruction quality compared to baselines at the same compression ratio.
Video variational autoencoders (VAEs) used in latent diffusion models typically require a sufficiently large number of latent channels to ensure high-quality video reconstruction. However, recent studies have revealed that an excessive number of latent channels can impede the convergence of latent diffusion models and deteriorate their generative performance, even when reconstruction quality remains high. We propose a latent compression method that removes high-frequency components in video latent representations rather than directly reducing the number of channels, which often compromises reconstruction fidelity. Experimental results demonstrate that the proposed method achieves superior video reconstruction quality compared to strong baselines while maintaining the same overall compression ratio.