Lower Dimensional Kernels for Video Discriminators
This addresses optimization challenges in video GANs for researchers and practitioners, offering an incremental improvement in efficiency and performance.
The paper tackles the difficulty of optimizing video GAN discriminators by showing that high kernel dimensions cause loss surfaces with high curvature, and proposes lower-dimensional video discriminators that double the performance of Temporal-GANs and achieve state-of-the-art results on datasets like UCF-101 on a single GPU.
This work presents an analysis of the discriminators used in Generative Adversarial Networks (GANs) for Video. We show that unconstrained video discriminator architectures induce a loss surface with high curvature which make optimisation difficult. We also show that this curvature becomes more extreme as the maximal kernel dimension of video discriminators increases. With these observations in hand, we propose a family of efficient Lower-Dimensional Video Discriminators for GANs (LDVD GANs). The proposed family of discriminators improve the performance of video GAN models they are applied to and demonstrate good performance on complex and diverse datasets such as UCF-101. In particular, we show that they can double the performance of Temporal-GANs and provide for state-of-the-art performance on a single GPU.