Mutual Suppression Network for Video Prediction using Disentangled Features
This addresses the problem of predicting future frames in videos for applications like autonomous driving or surveillance, though it appears incremental as it builds on existing encoder-decoder and adversarial training approaches.
The paper tackles video prediction by disentangling spatial and temporal features using a mutual suppression network (MSnet), which achieves better results than recent methods despite having simpler encoders.
Video prediction has been considered a difficult problem because the video contains not only high-dimensional spatial information but also complex temporal information. Video prediction can be performed by finding features in recent frames, and using them to generate approximations to upcoming frames. We approach this problem by disentangling spatial and temporal features in videos. We introduce a mutual suppression network (MSnet) which are trained in an adversarial manner and then produces spatial features which are free of motion information, and motion features with no spatial information. MSnet then uses motion-guided connection within an encoder-decoder-based architecture to transform spatial features from a previous frame to the time of an upcoming frame. We show how MSnet can be used for video prediction using disentangled representations. We also carry out experiments to assess the effectiveness of our method to disentangle features. MSnet obtains better results than other recent video prediction methods even though it has simpler encoders.