DidSee: Diffusion-Based Depth Completion for Material-Agnostic Robotic Perception and Manipulation
This work addresses a critical issue in robotic perception and manipulation by enhancing depth completion for non-Lambertian materials, though it appears incremental as it builds on existing diffusion models with specific modifications.
The paper tackled the problem of noisy and incomplete depth maps from RGB-D cameras for non-Lambertian objects by proposing DidSee, a diffusion-based depth completion framework that integrates a rescaled noise scheduler, noise-agnostic training, and a semantic enhancer, achieving state-of-the-art performance on multiple benchmarks and improving downstream robotic tasks.
Commercial RGB-D cameras often produce noisy, incomplete depth maps for non-Lambertian objects. Traditional depth completion methods struggle to generalize due to the limited diversity and scale of training data. Recent advances exploit visual priors from pre-trained text-to-image diffusion models to enhance generalization in dense prediction tasks. However, we find that biases arising from training-inference mismatches in the vanilla diffusion framework significantly impair depth completion performance. Additionally, the lack of distinct visual features in non-Lambertian regions further hinders precise prediction. To address these issues, we propose \textbf{DidSee}, a diffusion-based framework for depth completion on non-Lambertian objects. First, we integrate a rescaled noise scheduler enforcing a zero terminal signal-to-noise ratio to eliminate signal leakage bias. Second, we devise a noise-agnostic single-step training formulation to alleviate error accumulation caused by exposure bias and optimize the model with a task-specific loss. Finally, we incorporate a semantic enhancer that enables joint depth completion and semantic segmentation, distinguishing objects from backgrounds and yielding precise, fine-grained depth maps. DidSee achieves state-of-the-art performance on multiple benchmarks, demonstrates robust real-world generalization, and effectively improves downstream tasks such as category-level pose estimation and robotic grasping.