Towards Reliable Evaluation and Fast Training of Robust Semantic Segmentation Models
This addresses the reliability of adversarial robustness in semantic segmentation, a domain-specific task where prior methods were inadequate, offering practical improvements for researchers and practitioners in computer vision.
The paper tackles the problem of evaluating and training robust semantic segmentation models against adversarial attacks, revealing that existing attacks overestimate robustness and adversarial training methods are weak. It proposes new attacks and a method using robust ImageNet backbones to achieve robust models with up to six times faster training on datasets like PASCAL-VOC and ADE20k.
Adversarial robustness has been studied extensively in image classification, especially for the $\ell_\infty$-threat model, but significantly less so for related tasks such as object detection and semantic segmentation, where attacks turn out to be a much harder optimization problem than for image classification. We propose several problem-specific novel attacks minimizing different metrics in accuracy and mIoU. The ensemble of our attacks, SEA, shows that existing attacks severely overestimate the robustness of semantic segmentation models. Surprisingly, existing attempts of adversarial training for semantic segmentation models turn out to be weak or even completely non-robust. We investigate why previous adaptations of adversarial training to semantic segmentation failed and show how recently proposed robust ImageNet backbones can be used to obtain adversarially robust semantic segmentation models with up to six times less training time for PASCAL-VOC and the more challenging ADE20k. The associated code and robust models are available at https://github.com/nmndeep/robust-segmentation