Semantic-aligned Fusion Transformer for One-shot Object Detection
This work improves one-shot object detection for computer vision applications by enhancing feature fusion, though it is incremental as it builds on existing transformer-based methods.
The paper tackles the problem of one-shot object detection by addressing misaligned query-support semantics due to spatial and scale variances, proposing a Semantic-aligned Fusion Transformer (SaFT) that achieves significant performance gains and lifts state-of-the-art results on multiple benchmarks.
One-shot object detection aims at detecting novel objects according to merely one given instance. With extreme data scarcity, current approaches explore various feature fusions to obtain directly transferable meta-knowledge. Yet, their performances are often unsatisfactory. In this paper, we attribute this to inappropriate correlation methods that misalign query-support semantics by overlooking spatial structures and scale variances. Upon analysis, we leverage the attention mechanism and propose a simple but effective architecture named Semantic-aligned Fusion Transformer (SaFT) to resolve these issues. Specifically, we equip SaFT with a vertical fusion module (VFM) for cross-scale semantic enhancement and a horizontal fusion module (HFM) for cross-sample feature fusion. Together, they broaden the vision for each feature point from the support to a whole augmented feature pyramid from the query, facilitating semantic-aligned associations. Extensive experiments on multiple benchmarks demonstrate the superiority of our framework. Without fine-tuning on novel classes, it brings significant performance gains to one-stage baselines, lifting state-of-the-art results to a higher level.