Image to Sphere: Learning Equivariant Features for Efficient Pose Prediction
This addresses the challenge of efficient and accurate pose prediction for objects with symmetries in computer vision, offering a novel approach that improves sample efficiency and handles uncertainties.
The paper tackles the problem of predicting object pose from a single image by proposing a method that maps image features to the 3D rotation manifold using SO(3) equivariant layers, achieving state-of-the-art performance on the PASCAL3D+ dataset and effectively modeling object symmetries.
Predicting the pose of objects from a single image is an important but difficult computer vision problem. Methods that predict a single point estimate do not predict the pose of objects with symmetries well and cannot represent uncertainty. Alternatively, some works predict a distribution over orientations in $\mathrm{SO}(3)$. However, training such models can be computation- and sample-inefficient. Instead, we propose a novel mapping of features from the image domain to the 3D rotation manifold. Our method then leverages $\mathrm{SO}(3)$ equivariant layers, which are more sample efficient, and outputs a distribution over rotations that can be sampled at arbitrary resolution. We demonstrate the effectiveness of our method at object orientation prediction, and achieve state-of-the-art performance on the popular PASCAL3D+ dataset. Moreover, we show that our method can model complex object symmetries, without any modifications to the parameters or loss function. Code is available at https://dmklee.github.io/image2sphere.