State-Aware Tracker for Real-Time Video Object Segmentation
This work addresses the problem of real-time video object segmentation for applications requiring efficient and accurate tracking, representing an incremental improvement with a focus on speed-accuracy trade-offs.
The paper tackles semi-supervised video object segmentation by proposing a State-Aware Tracker (SAT) that leverages inter-frame consistency and feedback loops for self-adaptation, achieving 72.3% J&F mean accuracy at 39 FPS on the DAVIS2017-Val dataset.
In this work, we address the task of semi-supervised video object segmentation(VOS) and explore how to make efficient use of video property to tackle the challenge of semi-supervision. We propose a novel pipeline called State-Aware Tracker(SAT), which can produce accurate segmentation results with real-time speed. For higher efficiency, SAT takes advantage of the inter-frame consistency and deals with each target object as a tracklet. For more stable and robust performance over video sequences, SAT gets awareness for each state and makes self-adaptation via two feedback loops. One loop assists SAT in generating more stable tracklets. The other loop helps to construct a more robust and holistic target representation. SAT achieves a promising result of 72.3% J&F mean with 39 FPS on DAVIS2017-Val dataset, which shows a decent trade-off between efficiency and accuracy. Code will be released at github.com/MegviiDetection/video_analyst.