InMoDeGAN: Interpretable Motion Decomposition Generative Adversarial Network for Video Generation
This work provides a novel approach for researchers and developers in video generation to create more controllable and interpretable video synthesis models, addressing the challenge of understanding latent space in generative models.
This paper introduces InMoDeGAN, an unconditional video generative model that not only generates high-quality videos but also allows for the interpretation and manipulation of motion in its latent space. The model significantly outperforms state-of-the-art methods on the VoxCeleb2-mini and BAIR-robot datasets in terms of video quality.
In this work, we introduce an unconditional video generative model, InMoDeGAN, targeted to (a) generate high quality videos, as well as to (b) allow for interpretation of the latent space. For the latter, we place emphasis on interpreting and manipulating motion. Towards this, we decompose motion into semantic sub-spaces, which allow for control of generated samples. We design the architecture of InMoDeGAN-generator in accordance to proposed Linear Motion Decomposition, which carries the assumption that motion can be represented by a dictionary, with related vectors forming an orthogonal basis in the latent space. Each vector in the basis represents a semantic sub-space. In addition, a Temporal Pyramid Discriminator analyzes videos at different temporal resolutions. Extensive quantitative and qualitative analysis shows that our model systematically and significantly outperforms state-of-the-art methods on the VoxCeleb2-mini and BAIR-robot datasets w.r.t. video quality related to (a). Towards (b) we present experimental results, confirming that decomposed sub-spaces are interpretable and moreover, generated motion is controllable.