SeqCo-DETR: Sequence Consistency Training for Self-Supervised Object Detection with Transformers
This addresses the need for transformer-specific self-supervised methods in object detection, offering an incremental improvement over existing approaches.
The paper tackled the problem of self-supervised object detection with transformers by proposing SeqCo-DETR, which uses sequence consistency training and mask-based augmentation, achieving state-of-the-art results of 45.8 AP on MS COCO and 64.1 AP on PASCAL VOC.
Self-supervised pre-training and transformer-based networks have significantly improved the performance of object detection. However, most of the current self-supervised object detection methods are built on convolutional-based architectures. We believe that the transformers' sequence characteristics should be considered when designing a transformer-based self-supervised method for the object detection task. To this end, we propose SeqCo-DETR, a novel Sequence Consistency-based self-supervised method for object DEtection with TRansformers. SeqCo-DETR defines a simple but effective pretext by minimizes the discrepancy of the output sequences of transformers with different image views as input and leverages bipartite matching to find the most relevant sequence pairs to improve the sequence-level self-supervised representation learning performance. Furthermore, we provide a mask-based augmentation strategy incorporated with the sequence consistency strategy to extract more representative contextual information about the object for the object detection task. Our method achieves state-of-the-art results on MS COCO (45.8 AP) and PASCAL VOC (64.1 AP), demonstrating the effectiveness of our approach.