Discovery of Latent 3D Keypoints via End-to-end Geometric Reasoning
It addresses pose estimation for objects like cars and chairs, offering an unsupervised approach that improves over supervised methods.
The paper tackles 3D pose estimation by learning category-specific 3D keypoints from single images without ground-truth annotations, and it outperforms a fully supervised baseline on this task.
This paper presents KeypointNet, an end-to-end geometric reasoning framework to learn an optimal set of category-specific 3D keypoints, along with their detectors. Given a single image, KeypointNet extracts 3D keypoints that are optimized for a downstream task. We demonstrate this framework on 3D pose estimation by proposing a differentiable objective that seeks the optimal set of keypoints for recovering the relative pose between two views of an object. Our model discovers geometrically and semantically consistent keypoints across viewing angles and instances of an object category. Importantly, we find that our end-to-end framework using no ground-truth keypoint annotations outperforms a fully supervised baseline using the same neural network architecture on the task of pose estimation. The discovered 3D keypoints on the car, chair, and plane categories of ShapeNet are visualized at http://keypointnet.github.io/.