MobilePose: Real-Time Pose Estimation for Unseen Objects with Weak Shape Supervision
It addresses pose estimation for unseen objects, which is important for applications like augmented reality on mobile devices, but is incremental as it builds on existing single-shot solutions with efficiency improvements.
The paper tackles the problem of detecting unseen objects from RGB images and estimating their 3D poses, achieving real-time performance (e.g., 36 FPS on mobile devices) with higher accuracy and significantly smaller model sizes (2-3% of previous methods).
In this paper, we address the problem of detecting unseen objects from RGB images and estimating their poses in 3D. We propose two mobile friendly networks: MobilePose-Base and MobilePose-Shape. The former is used when there is only pose supervision, and the latter is for the case when shape supervision is available, even a weak one. We revisit shape features used in previous methods, including segmentation and coordinate map. We explain when and why pixel-level shape supervision can improve pose estimation. Consequently, we add shape prediction as an intermediate layer in the MobilePose-Shape, and let the network learn pose from shape. Our models are trained on mixed real and synthetic data, with weak and noisy shape supervision. They are ultra lightweight that can run in real-time on modern mobile devices (e.g. 36 FPS on Galaxy S20). Comparing with previous single-shot solutions, our method has higher accuracy, while using a significantly smaller model (2~3% in model size or number of parameters).