Physically-Based Photometric Bundle Adjustment in Non-Lambertian Environments
This addresses the reliability issue in camera pose and 3D geometry estimation for real-world scenes with non-diffuse reflections, which is an incremental improvement over existing methods.
The paper tackles the problem of photometric bundle adjustment failing in non-Lambertian environments by proposing a physically-based method with weights for material, illumination, and light path, and it outperforms existing approaches in accuracy as shown in experiments.
Photometric bundle adjustment (PBA) is widely used in estimating the camera pose and 3D geometry by assuming a Lambertian world. However, the assumption of photometric consistency is often violated since the non-diffuse reflection is common in real-world environments. The photometric inconsistency significantly affects the reliability of existing PBA methods. To solve this problem, we propose a novel physically-based PBA method. Specifically, we introduce the physically-based weights regarding material, illumination, and light path. These weights distinguish the pixel pairs with different levels of photometric inconsistency. We also design corresponding models for material estimation based on sequential images and illumination estimation based on point clouds. In addition, we establish the first SLAM-related dataset of non-Lambertian scenes with complete ground truth of illumination and material. Extensive experiments demonstrated that our PBA method outperforms existing approaches in accuracy.