ActiveZero: Mixed Domain Learning for Active Stereovision with Zero Annotation
This addresses the challenge of expensive real-world depth annotation for active stereovision systems, offering a practical solution for robotics and computer vision applications, though it builds incrementally on existing domain adaptation and self-supervised techniques.
The paper tackles the problem of obtaining accurate depth estimates in real-world active stereovision without real-world depth annotations by proposing ActiveZero, a mixed-domain learning framework that uses simulation data with supervision and real data with self-supervision. The result is state-of-the-art performance on real data, even surpassing a commercial depth sensor in evaluations.
Traditional depth sensors generate accurate real world depth estimates that surpass even the most advanced learning approaches trained only on simulation domains. Since ground truth depth is readily available in the simulation domain but quite difficult to obtain in the real domain, we propose a method that leverages the best of both worlds. In this paper we present a new framework, ActiveZero, which is a mixed domain learning solution for active stereovision systems that requires no real world depth annotation. First, we demonstrate the transferability of our method to out-of-distribution real data by using a mixed domain learning strategy. In the simulation domain, we use a combination of supervised disparity loss and self-supervised losses on a shape primitives dataset. By contrast, in the real domain, we only use self-supervised losses on a dataset that is out-of-distribution from either training simulation data or test real data. Second, our method introduces a novel self-supervised loss called temporal IR reprojection to increase the robustness and accuracy of our reprojections in hard-to-perceive regions. Finally, we show how the method can be trained end-to-end and that each module is important for attaining the end result. Extensive qualitative and quantitative evaluations on real data demonstrate state of the art results that can even beat a commercial depth sensor.