GPS-GLASS: Learning Nighttime Semantic Segmentation Using Daytime Video and GPS data
This addresses the problem of limited annotated nighttime data for autonomous driving systems, offering a novel approach to bridge the domain gap, though it is incremental in leveraging existing GPS and video data.
The paper tackles nighttime semantic segmentation for autonomous driving by proposing a GPS-based training framework that uses aligned daytime-nighttime image pairs and daytime video flow to generate pseudo supervision, enabling training without nighttime annotations, and demonstrates effectiveness on multiple datasets.
Semantic segmentation for autonomous driving should be robust against various in-the-wild environments. Nighttime semantic segmentation is especially challenging due to a lack of annotated nighttime images and a large domain gap from daytime images with sufficient annotation. In this paper, we propose a novel GPS-based training framework for nighttime semantic segmentation. Given GPS-aligned pairs of daytime and nighttime images, we perform cross-domain correspondence matching to obtain pixel-level pseudo supervision. Moreover, we conduct flow estimation between daytime video frames and apply GPS-based scaling to acquire another pixel-level pseudo supervision. Using these pseudo supervisions with a confidence map, we train a nighttime semantic segmentation network without any annotation from nighttime images. Experimental results demonstrate the effectiveness of the proposed method on several nighttime semantic segmentation datasets. Our source code is available at https://github.com/jimmy9704/GPS-GLASS.