Two-Level Temporal Relation Model for Online Video Instance Segmentation
This work addresses the problem of real-time video instance segmentation for applications requiring both accuracy and speed, representing a competitive but incremental advance in online methods.
The authors tackled the trade-off between quality and speed in Video Instance Segmentation by proposing an online method that matches offline performance, achieving state-of-the-art results on the YouTube-VIS dataset and demonstrating generalization to DAVIS for video object segmentation.
In Video Instance Segmentation (VIS), current approaches either focus on the quality of the results, by taking the whole video as input and processing it offline; or on speed, by handling it frame by frame at the cost of competitive performance. In this work, we propose an online method that is on par with the performance of the offline counterparts. We introduce a message-passing graph neural network that encodes objects and relates them through time. We additionally propose a novel module to fuse features from the feature pyramid network with residual connections. Our model, trained end-to-end, achieves state-of-the-art performance on the YouTube-VIS dataset within the online methods. Further experiments on DAVIS demonstrate the generalization capability of our model to the video object segmentation task. Code is available at: \url{https://github.com/caganselim/TLTM}