Semantic 3D Reconstruction with Continuous Regularization and Ray Potentials Using a Visibility Consistency Constraint
This work addresses the problem of accurate 3D reconstruction, including thin objects, for computer vision applications, representing an incremental advance with specific gains.
The paper tackles dense semantic 3D reconstruction by proposing a convex relaxation with a non-convex visibility constraint and a majorize-minimize optimization strategy, achieving new state-of-the-art results on two Middlebury datasets and improving over previous methods in qualitative evaluations.
We propose an approach for dense semantic 3D reconstruction which uses a data term that is defined as potentials over viewing rays, combined with continuous surface area penalization. Our formulation is a convex relaxation which we augment with a crucial non-convex constraint that ensures exact handling of visibility. To tackle the non-convex minimization problem, we propose a majorize-minimize type strategy which converges to a critical point. We demonstrate the benefits of using the non-convex constraint experimentally. For the geometry-only case, we set a new state of the art on two datasets of the commonly used Middlebury multi-view stereo benchmark. Moreover, our general-purpose formulation directly reconstructs thin objects, which are usually treated with specialized algorithms. A qualitative evaluation on the dense semantic 3D reconstruction task shows that we improve significantly over previous methods.