Projective Manifold Gradient Layer for Deep Rotation Regression
This addresses a key challenge in 3D vision and robotics for tasks requiring accurate rotation estimation, representing an incremental improvement over prior methods focused on representations.
The paper tackles the problem of regressing rotations on the SO(3) manifold with deep neural networks by proposing a manifold-aware gradient method to improve backpropagation, achieving new state-of-the-art performance in rotation estimation tasks.
Regressing rotations on SO(3) manifold using deep neural networks is an important yet unsolved problem. The gap between the Euclidean network output space and the non-Euclidean SO(3) manifold imposes a severe challenge for neural network learning in both forward and backward passes. While several works have proposed different regression-friendly rotation representations, very few works have been devoted to improving the gradient backpropagating in the backward pass. In this paper, we propose a manifold-aware gradient that directly backpropagates into deep network weights. Leveraging Riemannian optimization to construct a novel projective gradient, our proposed regularized projective manifold gradient (RPMG) method helps networks achieve new state-of-the-art performance in a variety of rotation estimation tasks. Our proposed gradient layer can also be applied to other smooth manifolds such as the unit sphere. Our project page is at https://jychen18.github.io/RPMG.