Time-rEversed diffusioN tEnsor Transformer: A new TENET of Few-Shot Object Detection
This work improves few-shot object detection for computer vision applications by introducing novel components to handle intra-class variations, though it builds incrementally on transformer-based approaches.
The paper tackles few-shot object detection by addressing information loss and lack of positional sensitivity in existing methods, proposing TENET and TRH to capture multi-way features and dynamic correlations, achieving state-of-the-art results on PASCAL VOC, FSOD, and COCO datasets.
In this paper, we tackle the challenging problem of Few-shot Object Detection. Existing FSOD pipelines (i) use average-pooled representations that result in information loss; and/or (ii) discard position information that can help detect object instances. Consequently, such pipelines are sensitive to large intra-class appearance and geometric variations between support and query images. To address these drawbacks, we propose a Time-rEversed diffusioN tEnsor Transformer (TENET), which i) forms high-order tensor representations that capture multi-way feature occurrences that are highly discriminative, and ii) uses a transformer that dynamically extracts correlations between the query image and the entire support set, instead of a single average-pooled support embedding. We also propose a Transformer Relation Head (TRH), equipped with higher-order representations, which encodes correlations between query regions and the entire support set, while being sensitive to the positional variability of object instances. Our model achieves state-of-the-art results on PASCAL VOC, FSOD, and COCO.