DIMOS: Disentangling Instance-level Moving Object Segmentation
This work addresses the challenge of segmenting small moving instances in multimodal MIS, a problem relevant to autonomous driving and surveillance, by improving feature density and cross-modal fusion.
DIMOS introduces a dual-disentangling feature extraction framework for multimodal moving instance segmentation that separates appearance and motion cues from event and image data, achieving state-of-the-art performance, particularly for small objects in fast motion and low-light conditions.
Moving instance segmentation (MIS) attracts increasing attention due to its broad applications in traffic surveillance, autonomous driving, and animal tracking. Event cameras record asynchronous brightness changes, providing high temporal resolution and dynamic range, which makes them highly sensitive to motion information. By fusing event and image features, motion cues from events can complement spatial details from images, enhancing the performance of MIS. However, current multimodal MIS methods still struggle to segment small moving instances, as event cameras often yield sparse features under limited resolution. Moreover, event features entangle appearance attributes with motion cues, which further restricts effective cross-modal fusion. To address these challenges, we first propose a dual-disentangling feature extraction framework that separates and extracts appearance and motion information within both image and event modalities, thereby improving feature density. Subsequently, a multi-granularity cross-modal alignment is introduced to align distributionally and semantically consistent features across modalities, enabling more effective fusion with rich spatial and temporal details. The experiment results demonstrate that our method achieves state-of-the-art performance in multimodal MIS, especially for small instances under challenging conditions such as fast motion and low-light settings.