Latest Object Memory Management for Temporally Consistent Video Instance Segmentation
This addresses the challenge of maintaining stable object identities in dynamic video scenes for applications like video analysis and autonomous systems, representing a strong specific gain rather than a foundational advancement.
The paper tackles the problem of temporally consistent video instance segmentation by introducing Latest Object Memory Management (LOMM), which improves long-term instance tracking and achieves a state-of-the-art AP score of 54.0 on the YouTube-VIS 2022 dataset.
In this paper, we present Latest Object Memory Management (LOMM) for temporally consistent video instance segmentation that significantly improves long-term instance tracking. At the core of our method is Latest Object Memory (LOM), which robustly tracks and continuously updates the latest states of objects by explicitly modeling their presence in each frame. This enables consistent tracking and accurate identity management across frames, enhancing both performance and reliability through the VIS process. Moreover, we introduce Decoupled Object Association (DOA), a strategy that separately handles newly appearing and already existing objects. By leveraging our memory system, DOA accurately assigns object indices, improving matching accuracy and ensuring stable identity consistency, even in dynamic scenes where objects frequently appear and disappear. Extensive experiments and ablation studies demonstrate the superiority of our method over traditional approaches, setting a new benchmark in VIS. Notably, our LOMM achieves state-of-the-art AP score of 54.0 on YouTube-VIS 2022, a dataset known for its challenging long videos. Project page: https://seung-hun-lee.github.io/projects/LOMM/