ICDepth: Taming Video Diffusion Models for Video Depth Estimation via In-Context Conditioning
It addresses the need for temporally consistent, geometrically accurate, and generalizable video depth estimation with high data efficiency, benefiting applications like 3D reconstruction and autonomous driving.
ICDepth adapts pre-trained text-to-video diffusion transformers for monocular video depth estimation via In-Context Conditioning, achieving state-of-the-art results on multiple benchmarks with only 0.8M training frames (6-13x less than competing generative methods) and strong zero-shot generalization.
Monocular video depth estimation requires temporal consistency, geometric accuracy, and generalization across diverse scenarios, yet existing methods struggle to achieve all three simultaneously. Discriminative models excel at per-frame accuracy but suffer from temporal drift due to limited context windows, while generative methods improve consistency and generalization at the cost of extensive training data (10M+ samples) and lack of geometric precision. In response to these issues, we introduce \textbf{ICDepth}, a framework that adapts pre-trained text-to-video diffusion transformers for video depth estimation via In-Context Conditioning (ICC), leveraging their rich spatial-temporal priors. To address key challenges in transferring ICC from generation to dense prediction, we propose: (1)~\textbf{SAND-Attention}, which ensures precise spatial-temporal alignment via shared RoPE and enforces unidirectional attention to prevent noise contamination; (2)~\textbf{SRFM}, which injects DINOv2 semantic and resolution priors to enhance geometric precision. ICDepth achieves state-of-the-art results on multiple benchmarks with remarkable data efficiency, trained on only 0.8M frames ($6$--$13\times$ less than competing generative methods), while demonstrating strong zero-shot generalization to diverse domains.