PaQ-DETR: Learning Pattern and Quality-Aware Dynamic Queries for Object Detection
This work addresses a specific bottleneck in DETR variants for object detection, offering incremental improvements in accuracy and interpretability.
The paper tackles the issues of fixed learnable queries and query utilization imbalance in DETR-based object detection, proposing PaQ-DETR to enhance query adaptivity and supervision balance, resulting in consistent gains of 1.5%-4.2% mAP across benchmarks like COCO and CityScapes.
Detection Transformer (DETR) has redefined object detection by casting it as a set prediction task within an end-to-end framework. Despite its elegance, DETR and its variants still rely on fixed learnable queries and suffer from severe query utilization imbalance, which limits adaptability and leaves the model capacity underused. We propose PaQ-DETR (Pattern and Quality-Aware DETR), a unified framework that enhances both query adaptivity and supervision balance. It learns a compact set of shared latent patterns capturing global semantics and dynamically generates image-specific queries through content-conditioned weighting. In parallel, a quality-aware one-to-many assignment strategy adaptively selects positive samples based on localizatio-classification consistency, enriching supervision and promoting balanced query optimization. Experiments on COCO, CityScapes, and other benchmarks show consistent gains of 1.5%-4.2% mAP across DETR backbones, including ResNet and Swin-Transformer. Beyond accuracy improvement, our method provides interpretable insights into how dynamic patterns cluster semantically across object categories.