Need for Speed: Zero-Shot Depth Completion with Single-Step Diffusion
This work addresses the efficiency gap for real-time 3D perception in robotics and autonomous systems, representing an incremental improvement over existing diffusion methods.
The authors tackled the problem of slow inference in diffusion-based depth completion by introducing Marigold-SSD, a single-step framework that eliminates test-time optimization, achieving significantly faster inference with a training cost of only 4.5 GPU days.
We introduce Marigold-SSD, a single-step, late-fusion depth completion framework that leverages strong diffusion priors while eliminating the costly test-time optimization typically associated with diffusion-based methods. By shifting computational burden from inference to finetuning, our approach enables efficient and robust 3D perception under real-world latency constraints. Marigold-SSD achieves significantly faster inference with a training cost of only 4.5 GPU days. We evaluate our method across four indoor and two outdoor benchmarks, demonstrating strong cross-domain generalization and zero-shot performance compared to existing depth completion approaches. Our approach significantly narrows the efficiency gap between diffusion-based and discriminative models. Finally, we challenge common evaluation protocols by analyzing performance under varying input sparsity levels. Page: https://dtu-pas.github.io/marigold-ssd/