Omni-DETR: Omni-Supervised Object Detection with Transformers
This work addresses the problem of reducing annotation costs for object detection in computer vision, enabling more efficient dataset creation, though it is incremental as it builds on existing student-teacher and transformer frameworks.
The paper tackles omni-supervised object detection by leveraging unlabeled, fully labeled, and weakly labeled annotations (e.g., image tags, counts, points) using a unified transformer-based architecture called Omni-DETR, which achieves state-of-the-art results on multiple datasets and shows that weak annotations improve detection performance with a better trade-off between annotation cost and accuracy than standard complete annotations.
We consider the problem of omni-supervised object detection, which can use unlabeled, fully labeled and weakly labeled annotations, such as image tags, counts, points, etc., for object detection. This is enabled by a unified architecture, Omni-DETR, based on the recent progress on student-teacher framework and end-to-end transformer based object detection. Under this unified architecture, different types of weak labels can be leveraged to generate accurate pseudo labels, by a bipartite matching based filtering mechanism, for the model to learn. In the experiments, Omni-DETR has achieved state-of-the-art results on multiple datasets and settings. And we have found that weak annotations can help to improve detection performance and a mixture of them can achieve a better trade-off between annotation cost and accuracy than the standard complete annotation. These findings could encourage larger object detection datasets with mixture annotations. The code is available at https://github.com/amazon-research/omni-detr.