DuCos: Duality Constrained Depth Super-Resolution via Foundation Model
This work addresses depth super-resolution for computer vision applications, offering a novel integration approach with foundation models.
The paper tackles depth super-resolution by proposing DuCos, a framework that integrates foundation models as prompts through Lagrangian duality theory, achieving superior accuracy, robustness, and generalization compared to state-of-the-art methods.
We introduce DuCos, a novel depth super-resolution framework grounded in Lagrangian duality theory, offering a flexible integration of multiple constraints and reconstruction objectives to enhance accuracy and robustness. Our DuCos is the first to significantly improve generalization across diverse scenarios with foundation models as prompts. The prompt design consists of two key components: Correlative Fusion (CF) and Gradient Regulation (GR). CF facilitates precise geometric alignment and effective fusion between prompt and depth features, while GR refines depth predictions by enforcing consistency with sharp-edged depth maps derived from foundation models. Crucially, these prompts are seamlessly embedded into the Lagrangian constraint term, forming a synergistic and principled framework. Extensive experiments demonstrate that DuCos outperforms existing state-of-the-art methods, achieving superior accuracy, robustness, and generalization.