From Snow to Rain: Evaluating Robustness, Calibration, and Complexity of Model-Based Robust Training
This addresses reliability challenges in safety-sensitive domains like autonomous driving, but is incremental as it builds on existing robust training methods.
The paper tackled robustness to natural corruptions like snow and rain in traffic sign recognition by evaluating model-based training approaches that use learned nuisance variation models, finding that these methods consistently outperformed baselines with model-based adversarial training providing the strongest robustness but higher computation, while model-based data augmentation achieved comparable robustness with less complexity.
Robustness to natural corruptions remains a critical challenge for reliable deep learning, particularly in safety-sensitive domains. We study a family of model-based training approaches that leverage a learned nuisance variation model to generate realistic corruptions, as well as new hybrid strategies that combine random coverage with adversarial refinement in nuisance space. Using the Challenging Unreal and Real Environments for Traffic Sign Recognition dataset (CURE-TSR), with Snow and Rain corruptions, we evaluate accuracy, calibration, and training complexity across corruption severities. Our results show that model-based methods consistently outperform baselines Vanilla, Adversarial Training, and AugMix baselines, with model-based adversarial training providing the strongest robustness under across all corruptions but at the expense of higher computation and model-based data augmentation achieving comparable robustness with $T$ less computational complexity without incurring a statistically significant drop in performance. These findings highlight the importance of learned nuisance models for capturing natural variability, and suggest a promising path toward more resilient and calibrated models under challenging conditions.