TSAR-MVS: Textureless-aware Segmentation and Correlative Refinement Guided Multi-View Stereo
This addresses a long-standing challenge in 3D reconstruction for computer vision applications, but it is incremental as it builds on existing MVS methods with specific enhancements.
The paper tackles the problem of reconstructing textureless areas in multi-view stereo by proposing TSAR-MVS, which uses filtering, refinement, and segmentation to improve depth estimation, achieving superior performance on ETH3D, Tanks & Temples, and Strecha datasets.
The reconstruction of textureless areas has long been a challenging problem in MVS due to lack of reliable pixel correspondences between images. In this paper, we propose the Textureless-aware Segmentation And Correlative Refinement guided Multi-View Stereo (TSAR-MVS), a novel method that effectively tackles challenges posed by textureless areas in 3D reconstruction through filtering, refinement and segmentation. First, we implement the joint hypothesis filtering, a technique that merges a confidence estimator with a disparity discontinuity detector to eliminate incorrect depth estimations. Second, to spread the pixels with confident depth, we introduce an iterative correlation refinement strategy that leverages RANSAC to generate 3D planes based on superpixels, succeeded by a weighted median filter for broadening the influence of accurately determined pixels. Finally, we present a textureless-aware segmentation method that leverages edge detection and line detection for accurately identify large textureless regions for further depth completion. Experiments on ETH3D, Tanks & Temples and Strecha datasets demonstrate the superior performance and strong generalization capability of our proposed method.