VIDM: Video Implicit Diffusion Models
This work addresses video synthesis for applications like entertainment and simulation, representing an incremental advance by adapting diffusion models to video with specific optimizations.
The paper tackles video generation by proposing a diffusion model that implicitly models motion through latent frame features, achieving significant improvements in FVD scores and visual quality over GAN-based methods.
Diffusion models have emerged as a powerful generative method for synthesizing high-quality and diverse set of images. In this paper, we propose a video generation method based on diffusion models, where the effects of motion are modeled in an implicit condition manner, i.e. one can sample plausible video motions according to the latent feature of frames. We improve the quality of the generated videos by proposing multiple strategies such as sampling space truncation, robustness penalty, and positional group normalization. Various experiments are conducted on datasets consisting of videos with different resolutions and different number of frames. Results show that the proposed method outperforms the state-of-the-art generative adversarial network-based methods by a significant margin in terms of FVD scores as well as perceptible visual quality.