Self-supervised Event-based Monocular Depth Estimation using Cross-modal Consistency
This work addresses the annotation cost issue for depth estimation in event cameras, which is useful for applications in high-speed or challenging lighting conditions, but it is incremental as it builds on existing self-supervised and event-based methods.
The paper tackles the problem of costly depth annotation for event cameras by proposing a self-supervised monocular depth estimation framework called EMoDepth, which uses cross-modal consistency from intensity frames during training and achieves accuracy that outperforms existing supervised event-based and unsupervised frame-based methods on MVSEC and DSEC datasets.
An event camera is a novel vision sensor that can capture per-pixel brightness changes and output a stream of asynchronous ``events''. It has advantages over conventional cameras in those scenes with high-speed motions and challenging lighting conditions because of the high temporal resolution, high dynamic range, low bandwidth, low power consumption, and no motion blur. Therefore, several supervised monocular depth estimation from events is proposed to address scenes difficult for conventional cameras. However, depth annotation is costly and time-consuming. In this paper, to lower the annotation cost, we propose a self-supervised event-based monocular depth estimation framework named EMoDepth. EMoDepth constrains the training process using the cross-modal consistency from intensity frames that are aligned with events in the pixel coordinate. Moreover, in inference, only events are used for monocular depth prediction. Additionally, we design a multi-scale skip-connection architecture to effectively fuse features for depth estimation while maintaining high inference speed. Experiments on MVSEC and DSEC datasets demonstrate that our contributions are effective and that the accuracy can outperform existing supervised event-based and unsupervised frame-based methods.