Mitigating Query Selection Bias in Referring Video Object Segmentation
This addresses a specific bottleneck in RVOS for video analysis applications, representing an incremental advance with novel method design.
The paper tackles query selection bias in Referring Video Object Segmentation (RVOS), where static queries are misled by distractors, by proposing Triple Query Former (TQF) that factorizes queries into appearance, intra-frame interaction, and inter-frame motion components, achieving improved performance on multiple benchmarks.
Recently, query-based methods have achieved remarkable performance in Referring Video Object Segmentation (RVOS) by using textual static object queries to drive cross-modal alignment. However, these static queries are easily misled by distractors with similar appearance or motion, resulting in \emph{query selection bias}. To address this issue, we propose Triple Query Former (TQF), which factorizes the referring query into three specialized components: an appearance query for static attributes, an intra-frame interaction query for spatial relations, and an inter-frame motion query for temporal association. Instead of relying solely on textual embeddings, our queries are dynamically constructed by integrating both linguistic cues and visual guidance. Furthermore, we introduce two motion-aware aggregation modules that enhance object token representations: Intra-frame Interaction Aggregation incorporates position-aware interactions among objects within a single frame, while Inter-frame Motion Aggregation leverages trajectory-guided alignment across frames to ensure temporal coherence. Extensive experiments on multiple RVOS benchmarks demonstrate the advantages of TQF and the effectiveness of our structured query design and motion-aware aggregation modules.