Detection Hub: Unifying Object Detection Datasets via Query Adaptation on Language Embedding
This addresses the challenge of leveraging large-scale data for improved object detection, which is incremental as it builds on existing methods to handle dataset inconsistencies.
The paper tackles the problem of inconsistent taxonomies and domain gaps when combining multiple object detection datasets, proposing Detection Hub to unify them via dataset-aware query adaptation and semantic alignment using language embeddings, achieving significant performance gains and state-of-the-art results on the UODB benchmark.
Combining multiple datasets enables performance boost on many computer vision tasks. But similar trend has not been witnessed in object detection when combining multiple datasets due to two inconsistencies among detection datasets: taxonomy difference and domain gap. In this paper, we address these challenges by a new design (named Detection Hub) that is dataset-aware and category-aligned. It not only mitigates the dataset inconsistency but also provides coherent guidance for the detector to learn across multiple datasets. In particular, the dataset-aware design is achieved by learning a dataset embedding that is used to adapt object queries as well as convolutional kernels in detection heads. The categories across datasets are semantically aligned into a unified space by replacing one-hot category representations with word embedding and leveraging the semantic coherence of language embedding. Detection Hub fulfills the benefits of large data on object detection. Experiments demonstrate that joint training on multiple datasets achieves significant performance gains over training on each dataset alone. Detection Hub further achieves SoTA performance on UODB benchmark with wide variety of datasets.