Hierarchical Patch VAE-GAN: Generating Diverse Videos from a Single Sample
This addresses the challenge of limited training data for video generation, enabling more varied content creation from minimal input, though it is incremental as it builds on prior patch-GAN approaches.
The paper tackles the problem of generating diverse videos from a single sample, where existing methods often collapse into generating similar outputs, and introduces a hierarchical patch-based VAE-GAN that achieves high diversity and quality in video generation.
We consider the task of generating diverse and novel videos from a single video sample. Recently, new hierarchical patch-GAN based approaches were proposed for generating diverse images, given only a single sample at training time. Moving to videos, these approaches fail to generate diverse samples, and often collapse into generating samples similar to the training video. We introduce a novel patch-based variational autoencoder (VAE) which allows for a much greater diversity in generation. Using this tool, a new hierarchical video generation scheme is constructed: at coarse scales, our patch-VAE is employed, ensuring samples are of high diversity. Subsequently, at finer scales, a patch-GAN renders the fine details, resulting in high quality videos. Our experiments show that the proposed method produces diverse samples in both the image domain, and the more challenging video domain.