Multi-Frame GAN: Image Enhancement for Stereo Visual Odometry in Low Light
This addresses low-light navigation challenges for robotics or autonomous systems, but it is incremental as it builds on existing GAN and visual odometry techniques.
The paper tackles the problem of stereo visual odometry in low light by proposing a multi-frame GAN for image sequence enhancement, resulting in improved performance for state-of-the-art methods on synthetic and real datasets.
We propose the concept of a multi-frame GAN (MFGAN) and demonstrate its potential as an image sequence enhancement for stereo visual odometry in low light conditions. We base our method on an invertible adversarial network to transfer the beneficial features of brightly illuminated scenes to the sequence in poor illumination without costly paired datasets. In order to preserve the coherent geometric cues for the translated sequence, we present a novel network architecture as well as a novel loss term combining temporal and stereo consistencies based on optical flow estimation. We demonstrate that the enhanced sequences improve the performance of state-of-the-art feature-based and direct stereo visual odometry methods on both synthetic and real datasets in challenging illumination. We also show that MFGAN outperforms other state-of-the-art image enhancement and style transfer methods by a large margin in terms of visual odometry.