Adversarial Video Generation on Complex Datasets
It advances video modeling for applications like entertainment and surveillance by producing higher fidelity videos, though it is incremental as it builds on existing GAN scaling methods.
The paper tackles video generation by scaling up Generative Adversarial Networks on the complex Kinetics-600 dataset, achieving state-of-the-art Fréchet Inception Distance for video prediction on Kinetics-600 and Inception Score for video synthesis on UCF-101.
Generative models of natural images have progressed towards high fidelity samples by the strong leveraging of scale. We attempt to carry this success to the field of video modeling by showing that large Generative Adversarial Networks trained on the complex Kinetics-600 dataset are able to produce video samples of substantially higher complexity and fidelity than previous work. Our proposed model, Dual Video Discriminator GAN (DVD-GAN), scales to longer and higher resolution videos by leveraging a computationally efficient decomposition of its discriminator. We evaluate on the related tasks of video synthesis and video prediction, and achieve new state-of-the-art Fréchet Inception Distance for prediction for Kinetics-600, as well as state-of-the-art Inception Score for synthesis on the UCF-101 dataset, alongside establishing a strong baseline for synthesis on Kinetics-600.