CHOSEN: Contrastive Hypothesis Selection for Multi-View Depth Refinement
This work addresses depth refinement for multi-view stereo systems, offering a flexible solution that can be integrated into various pipelines, though it appears incremental as it builds upon existing methods.
The authors tackled the problem of refining depth estimates in multi-view stereo systems by proposing CHOSEN, a framework that iteratively selects the best depth hypotheses using contrastive learning, resulting in improved depth and normal accuracy compared to existing deep learning pipelines.
We propose CHOSEN, a simple yet flexible, robust and effective multi-view depth refinement framework. It can be employed in any existing multi-view stereo pipeline, with straightforward generalization capability for different multi-view capture systems such as camera relative positioning and lenses. Given an initial depth estimation, CHOSEN iteratively re-samples and selects the best hypotheses, and automatically adapts to different metric or intrinsic scales determined by the capture system. The key to our approach is the application of contrastive learning in an appropriate solution space and a carefully designed hypothesis feature, based on which positive and negative hypotheses can be effectively distinguished. Integrated in a simple baseline multi-view stereo pipeline, CHOSEN delivers impressive quality in terms of depth and normal accuracy compared to many current deep learning based multi-view stereo pipelines.