FFB6D: A Full Flow Bidirectional Fusion Network for 6D Pose Estimation
This work addresses a domain-specific problem in computer vision for robotics and augmented reality, offering an incremental improvement by enhancing fusion techniques for existing data sources.
The paper tackles the problem of 6D pose estimation from a single RGBD image by proposing FFB6D, a network that fully leverages complementary appearance and geometry information through bidirectional fusion and a keypoint selection algorithm, resulting in state-of-the-art performance with large margins on benchmarks.
In this work, we present FFB6D, a Full Flow Bidirectional fusion network designed for 6D pose estimation from a single RGBD image. Our key insight is that appearance information in the RGB image and geometry information from the depth image are two complementary data sources, and it still remains unknown how to fully leverage them. Towards this end, we propose FFB6D, which learns to combine appearance and geometry information for representation learning as well as output representation selection. Specifically, at the representation learning stage, we build bidirectional fusion modules in the full flow of the two networks, where fusion is applied to each encoding and decoding layer. In this way, the two networks can leverage local and global complementary information from the other one to obtain better representations. Moreover, at the output representation stage, we designed a simple but effective 3D keypoints selection algorithm considering the texture and geometry information of objects, which simplifies keypoint localization for precise pose estimation. Experimental results show that our method outperforms the state-of-the-art by large margins on several benchmarks. Code and video are available at \url{https://github.com/ethnhe/FFB6D.git}.