Lookahead Adversarial Learning for Near Real-Time Semantic Segmentation
This work addresses the problem of improving near real-time semantic segmentation for applications like autonomous driving or robotics, though it is incremental as it builds on existing adversarial and segmentation methods.
The paper tackles the challenge of integrating state-of-the-art semantic segmentation models into adversarial learning settings, which often suffer from convergence and stability issues, by proposing a lookahead adversarial learning approach with a label map aggregation module, resulting in performance improvements of up to +5% in some classes on three standard datasets.
Semantic segmentation is one of the most fundamental problems in computer vision with significant impact on a wide variety of applications. Adversarial learning is shown to be an effective approach for improving semantic segmentation quality by enforcing higher-level pixel correlations and structural information. However, state-of-the-art semantic segmentation models cannot be easily plugged into an adversarial setting because they are not designed to accommodate convergence and stability issues in adversarial networks. We bridge this gap by building a conditional adversarial network with a state-of-the-art segmentation model (DeepLabv3+) at its core. To battle the stability issues, we introduce a novel lookahead adversarial learning (LoAd) approach with an embedded label map aggregation module. We focus on semantic segmentation models that run fast at inference for near real-time field applications. Through extensive experimentation, we demonstrate that the proposed solution can alleviate divergence issues in an adversarial semantic segmentation setting and results in considerable performance improvements (+5% in some classes) on the baseline for three standard datasets.