Deep Generative Networks For Sequence Prediction
This work addresses sequence prediction problems for high-dimensional input data, but it appears incremental as it builds on existing GSN frameworks with new model variants.
The thesis tackled unsupervised time series representation learning for sequence prediction by decoupling static input from recurrent sequence representations using Generative Stochastic Networks (GSNs), showing that GSNs can learn useful representations for complex sequential data like MNIST digits, bouncing ball videos, and motion capture data.
This thesis investigates unsupervised time series representation learning for sequence prediction problems, i.e. generating nice-looking input samples given a previous history, for high dimensional input sequences by decoupling the static input representation from the recurrent sequence representation. We introduce three models based on Generative Stochastic Networks (GSN) for unsupervised sequence learning and prediction. Experimental results for these three models are presented on pixels of sequential handwritten digit (MNIST) data, videos of low-resolution bouncing balls, and motion capture data. The main contribution of this thesis is to provide evidence that GSNs are a viable framework to learn useful representations of complex sequential input data, and to suggest a new framework for deep generative models to learn complex sequences by decoupling static input representations from dynamic time dependency representations.