Video Propagation Networks
This work addresses video processing tasks such as segmentation and color propagation, offering a generalizable technique that is incremental in improving efficiency and performance.
The paper tackles the problem of propagating structured information like semantic labels through video frames by proposing a Video Propagation Network that processes frames adaptively and online without needing future frames, achieving increased performance compared to previous task-specific methods with favorable runtime.
We propose a technique that propagates information forward through video data. The method is conceptually simple and can be applied to tasks that require the propagation of structured information, such as semantic labels, based on video content. We propose a 'Video Propagation Network' that processes video frames in an adaptive manner. The model is applied online: it propagates information forward without the need to access future frames. In particular we combine two components, a temporal bilateral network for dense and video adaptive filtering, followed by a spatial network to refine features and increased flexibility. We present experiments on video object segmentation and semantic video segmentation and show increased performance comparing to the best previous task-specific methods, while having favorable runtime. Additionally we demonstrate our approach on an example regression task of color propagation in a grayscale video.