TransPoser: Transformer as an Optimizer for Joint Object Shape and Pose Estimation
This addresses the problem of 3D object understanding for robotics and AR/VR applications, presenting a novel neural optimization approach that is more efficient than traditional methods.
The paper tackles joint shape and pose estimation of rigid objects from RGB-D images by introducing DeepDDF for shape representation and TransPoser, a Transformer-based optimizer, achieving state-of-the-art accuracy efficiently in experiments on synthetic and real data.
We propose a novel method for joint estimation of shape and pose of rigid objects from their sequentially observed RGB-D images. In sharp contrast to past approaches that rely on complex non-linear optimization, we propose to formulate it as a neural optimization that learns to efficiently estimate the shape and pose. We introduce Deep Directional Distance Function (DeepDDF), a neural network that directly outputs the depth image of an object given the camera viewpoint and viewing direction, for efficient error computation in 2D image space. We formulate the joint estimation itself as a Transformer which we refer to as TransPoser. We fully leverage the tokenization and multi-head attention to sequentially process the growing set of observations and to efficiently update the shape and pose with a learned momentum, respectively. Experimental results on synthetic and real data show that DeepDDF achieves high accuracy as a category-level object shape representation and TransPoser achieves state-of-the-art accuracy efficiently for joint shape and pose estimation.