DTSGAN: Learning Dynamic Textures via Spatiotemporal Generative Adversarial Network
This addresses the problem of generating realistic video textures for applications in computer graphics and vision, but it is incremental as it builds on existing GAN methods with specific improvements.
The paper tackles dynamic texture synthesis from a single reference video by introducing DTSGAN, a spatiotemporal generative adversarial network that captures motion and content distribution, generating sequences from coarse to fine scales with a strategy to avoid mode collapse; experiments show it produces high-quality textures and natural motion.
Dynamic texture synthesis aims to generate sequences that are visually similar to a reference video texture and exhibit specific stationary properties in time. In this paper, we introduce a spatiotemporal generative adversarial network (DTSGAN) that can learn from a single dynamic texture by capturing its motion and content distribution. With the pipeline of DTSGAN, a new video sequence is generated from the coarsest scale to the finest one. To avoid mode collapse, we propose a novel strategy for data updates that helps improve the diversity of generated results. Qualitative and quantitative experiments show that our model is able to generate high quality dynamic textures and natural motion.