NightLab: A Dual-level Architecture with Hardness Detection for Segmentation at Night
This addresses the understudied problem of nighttime segmentation for autonomous driving, representing a strong domain-specific advancement.
The paper tackles nighttime semantic segmentation, a challenging problem for applications like self-driving cars, by proposing NightLab, a dual-level architecture with hardness detection that achieves state-of-the-art performance on NightCity and BDD100K datasets.
The semantic segmentation of nighttime scenes is a challenging problem that is key to impactful applications like self-driving cars. Yet, it has received little attention compared to its daytime counterpart. In this paper, we propose NightLab, a novel nighttime segmentation framework that leverages multiple deep learning models imbued with night-aware features to yield State-of-The-Art (SoTA) performance on multiple night segmentation benchmarks. Notably, NightLab contains models at two levels of granularity, i.e. image and regional, and each level is composed of light adaptation and segmentation modules. Given a nighttime image, the image level model provides an initial segmentation estimate while, in parallel, a hardness detection module identifies regions and their surrounding context that need further analysis. A regional level model focuses on these difficult regions to provide a significantly improved segmentation. All the models in NightLab are trained end-to-end using a set of proposed night-aware losses without handcrafted heuristics. Extensive experiments on the NightCity and BDD100K datasets show NightLab achieves SoTA performance compared to concurrent methods.