Adaptive Stereo Depth Estimation with Multi-Spectral Images Across All Lighting Conditions
This work addresses depth estimation for applications like autonomous driving or robotics in poor lighting, though it appears incremental as it builds on existing multi-spectral methods.
The paper tackles the challenge of depth estimation under adverse lighting conditions by proposing a novel framework that integrates visible light and thermal images as a stereo pair, achieving state-of-the-art performance on the Multi-Spectral Stereo dataset.
Depth estimation under adverse conditions remains a significant challenge. Recently, multi-spectral depth estimation, which integrates both visible light and thermal images, has shown promise in addressing this issue. However, existing algorithms struggle with precise pixel-level feature matching, limiting their ability to fully exploit geometric constraints across different spectra. To address this, we propose a novel framework incorporating stereo depth estimation to enforce accurate geometric constraints. In particular, we treat the visible light and thermal images as a stereo pair and utilize a Cross-modal Feature Matching (CFM) Module to construct a cost volume for pixel-level matching. To mitigate the effects of poor lighting on stereo matching, we introduce Degradation Masking, which leverages robust monocular thermal depth estimation in degraded regions. Our method achieves state-of-the-art (SOTA) performance on the Multi-Spectral Stereo (MS2) dataset, with qualitative evaluations demonstrating high-quality depth maps under varying lighting conditions.