Learning Dynamic Network Using a Reuse Gate Function in Semi-supervised Video Object Segmentation
This work offers an incremental improvement in computational efficiency for existing Semi-VOS methods, benefiting users who require faster processing without significant accuracy loss.
This paper addresses the computational redundancy in semi-supervised video object segmentation (Semi-VOS) for stationary or slow-moving objects. It proposes a dynamic network that estimates inter-frame change and selectively reuses features from previous frames, significantly improving inference speed on DAVIS 16, DAVIS 17, and YouTube-VOS datasets with minimal accuracy degradation.
Current state-of-the-art approaches for Semi-supervised Video Object Segmentation (Semi-VOS) propagates information from previous frames to generate segmentation mask for the current frame. This results in high-quality segmentation across challenging scenarios such as changes in appearance and occlusion. But it also leads to unnecessary computations for stationary or slow-moving objects where the change across frames is minimal. In this work, we exploit this observation by using temporal information to quickly identify frames with minimal change and skip the heavyweight mask generation step. To realize this efficiency, we propose a novel dynamic network that estimates change across frames and decides which path -- computing a full network or reusing previous frame's feature -- to choose depending on the expected similarity. Experimental results show that our approach significantly improves inference speed without much accuracy degradation on challenging Semi-VOS datasets -- DAVIS 16, DAVIS 17, and YouTube-VOS. Furthermore, our approach can be applied to multiple Semi-VOS methods demonstrating its generality. The code is available in https://github.com/HYOJINPARK/Reuse_VOS.