GoMVS: Geometrically Consistent Cost Aggregation for Multi-View Stereo
This work addresses a key bottleneck in 3D reconstruction for computer vision applications, offering a novel method that improves accuracy in multi-view stereo.
The paper tackles the problem of geometric inconsistency in cost aggregation for multi-view stereo networks by proposing GoMVS, which uses a geometric consistent propagation module to correspond and propagate adjacent costs based on local smoothness and surface normals, achieving state-of-the-art performance with top rankings on benchmarks like Tanks & Temple Advanced.
Matching cost aggregation plays a fundamental role in learning-based multi-view stereo networks. However, directly aggregating adjacent costs can lead to suboptimal results due to local geometric inconsistency. Related methods either seek selective aggregation or improve aggregated depth in the 2D space, both are unable to handle geometric inconsistency in the cost volume effectively. In this paper, we propose GoMVS to aggregate geometrically consistent costs, yielding better utilization of adjacent geometries. More specifically, we correspond and propagate adjacent costs to the reference pixel by leveraging the local geometric smoothness in conjunction with surface normals. We achieve this by the geometric consistent propagation (GCP) module. It computes the correspondence from the adjacent depth hypothesis space to the reference depth space using surface normals, then uses the correspondence to propagate adjacent costs to the reference geometry, followed by a convolution for aggregation. Our method achieves new state-of-the-art performance on DTU, Tanks & Temple, and ETH3D datasets. Notably, our method ranks 1st on the Tanks & Temple Advanced benchmark.