OMNI-DC: Highly Robust Depth Completion with Multiresolution Depth Integration
This addresses robustness issues in depth completion for real-world applications, though it appears incremental as it builds on existing methods with novel components.
The paper tackles the problem of poor generalization in depth completion to new datasets or sparse depth patterns by proposing OMNI-DC, which reduces errors by up to 43% across 7 datasets in zero-shot settings.
Depth completion (DC) aims to predict a dense depth map from an RGB image and a sparse depth map. Existing DC methods generalize poorly to new datasets or unseen sparse depth patterns, limiting their real-world applications. We propose OMNI-DC, a highly robust DC model that generalizes well zero-shot to various datasets. The key design is a novel Multi-resolution Depth Integrator, allowing our model to deal with very sparse depth inputs. We also introduce a novel Laplacian loss to model the ambiguity in the training process. Moreover, we train OMNI-DC on a mixture of high-quality datasets with a scale normalization technique and synthetic depth patterns. Extensive experiments on 7 datasets show consistent improvements over baselines, reducing errors by as much as 43%. Codes and checkpoints are available at https://github.com/princeton-vl/OMNI-DC.