Symmetry Aware Evaluation of 3D Object Detection and Pose Estimation in Scenes of Many Parts in Bulk
This work addresses evaluation challenges for researchers in 3D computer vision, though it is incremental in improving existing protocols and datasets.
The paper tackles the problems of limited annotated data and inadequate evaluation protocols for 3D object detection and pose estimation, particularly for symmetric objects, by creating a large-scale dataset of thousands of scenes with automatic annotation and proposing a symmetry-aware evaluation methodology that leads to significant performance gains.
While 3D object detection and pose estimation has been studied for a long time, its evaluation is not yet completely satisfactory. Indeed, existing datasets typically consist in numerous acquisitions of only a few scenes because of the tediousness of pose annotation, and existing evaluation protocols cannot handle properly objects with symmetries. This work aims at addressing those two points. We first present automatic techniques to produce fully annotated RGBD data of many object instances in arbitrary poses, with which we produce a dataset of thousands of independent scenes of bulk parts composed of both real and synthetic images. We then propose a consistent evaluation methodology suitable for any rigid object, regardless of its symmetries. We illustrate it with two reference object detection and pose estimation methods on different objects, and show that incorporating symmetry considerations into pose estimation methods themselves can lead to significant performance gains. The proposed dataset is available at http://rbregier.github.io/dataset2017.