Learning Temporal Transformations From Time-Lapse Videos
This addresses the challenge of enabling computers to visualize object transformations over time, which is incremental as it builds on existing generative modeling techniques.
The paper tackles the problem of learning computational models to predict future appearances of objects from time-lapse videos, using generative models for tasks like single-frame and multi-frame prediction, with results evaluated qualitatively, quantitatively, and through human assessments.
Based on life-long observations of physical, chemical, and biologic phenomena in the natural world, humans can often easily picture in their minds what an object will look like in the future. But, what about computers? In this paper, we learn computational models of object transformations from time-lapse videos. In particular, we explore the use of generative models to create depictions of objects at future times. These models explore several different prediction tasks: generating a future state given a single depiction of an object, generating a future state given two depictions of an object at different times, and generating future states recursively in a recurrent framework. We provide both qualitative and quantitative evaluations of the generated results, and also conduct a human evaluation to compare variations of our models.