A Study of Posterior Stability for Time-Series Latent Diffusion
This addresses a specific technical bottleneck (posterior collapse) for researchers working on time-series generation with latent diffusion models.
The paper tackles the problem of posterior collapse in time-series latent diffusion models, which reduces them to less expressive VAEs, and introduces a new framework with stable posterior that significantly outperforms previous baselines in time series synthesis.
Latent diffusion has demonstrated promising results in image generation and permits efficient sampling. However, this framework might suffer from the problem of posterior collapse when applied to time series. In this paper, we first show that posterior collapse will reduce latent diffusion to a variational autoencoder (VAE), making it less expressive. This highlights the importance of addressing this issue. We then introduce a principled method: dependency measure, that quantifies the sensitivity of a recurrent decoder to input variables. Using this tool, we confirm that posterior collapse significantly affects time-series latent diffusion on real datasets, and a phenomenon termed dependency illusion is also discovered in the case of shuffled time series. Finally, building on our theoretical and empirical studies, we introduce a new framework that extends latent diffusion and has a stable posterior. Extensive experiments on multiple real time-series datasets show that our new framework is free from posterior collapse and significantly outperforms previous baselines in time series synthesis.