Memory Matching is not Enough: Jointly Improving Memory Matching and Decoding for Video Object Segmentation
This work addresses a critical bottleneck in video object segmentation for computer vision applications, offering incremental improvements over existing memory-based methods.
The paper tackles the problem of false matching and information loss in memory-based video object segmentation by jointly improving memory matching and decoding stages, achieving state-of-the-art performance on benchmarks like DAVIS 2016 (92.4%), DAVIS 2017 (88.1%), and YouTubeVOS 2018 (84.8%).
Memory-based video object segmentation methods model multiple objects over long temporal-spatial spans by establishing memory bank, which achieve the remarkable performance. However, they struggle to overcome the false matching and are prone to lose critical information, resulting in confusion among different objects. In this paper, we propose an effective approach which jointly improving the matching and decoding stages to alleviate the false matching issue.For the memory matching stage, we present a cost aware mechanism that suppresses the slight errors for short-term memory and a shunted cross-scale matching for long-term memory which establish a wide filed matching spaces for various object scales. For the readout decoding stage, we implement a compensatory mechanism aims at recovering the essential information where missing at the matching stage. Our approach achieves the outstanding performance in several popular benchmarks (i.e., DAVIS 2016&2017 Val (92.4%&88.1%), and DAVIS 2017 Test (83.9%)), and achieves 84.8%&84.6% on YouTubeVOS 2018&2019 Val.