Cross-domain Few-shot Object Detection with Multi-modal Textual Enrichment
This work addresses domain adaptation challenges in few-shot object detection for applications requiring cross-modal integration, representing an incremental advancement in leveraging textual semantics.
The paper tackles the problem of performance degradation in multi-modal object detection under domain shifts by incorporating rich textual information to establish robust knowledge relationships between visual instances and language descriptions, resulting in significant improvements over existing few-shot object detection approaches on common benchmarks.
Advancements in cross-modal feature extraction and integration have significantly enhanced performance in few-shot learning tasks. However, current multi-modal object detection (MM-OD) methods often experience notable performance degradation when encountering substantial domain shifts. We propose that incorporating rich textual information can enable the model to establish a more robust knowledge relationship between visual instances and their corresponding language descriptions, thereby mitigating the challenges of domain shift. Specifically, we focus on the problem of Cross-Domain Multi-Modal Few-Shot Object Detection (CDMM-FSOD) and introduce a meta-learning-based framework designed to leverage rich textual semantics as an auxiliary modality to achieve effective domain adaptation. Our new architecture incorporates two key components: (i) A multi-modal feature aggregation module, which aligns visual and linguistic feature embeddings to ensure cohesive integration across modalities. (ii) A rich text semantic rectification module, which employs bidirectional text feature generation to refine multi-modal feature alignment, thereby enhancing understanding of language and its application in object detection. We evaluate the proposed method on common cross-domain object detection benchmarks and demonstrate that it significantly surpasses existing few-shot object detection approaches.