A Validation Strategy for Deep Learning Models: Evaluating and Enhancing Robustness
This work addresses the reliability issue for deep learning practitioners by providing a targeted validation method to improve robustness against perturbations, though it is incremental as it builds on existing robustness validation techniques.
The paper tackles the problem of deep learning models being vulnerable to data distortions like adversarial and common corruption perturbations, which degrade performance and challenge reliability, by proposing a validation approach that extracts 'weak robust' samples from training data to identify vulnerabilities and enhance robustness, demonstrating effectiveness on CIFAR-10, CIFAR-100, and ImageNet datasets with improvements in model reliability.
Data-driven models, especially deep learning classifiers often demonstrate great success on clean datasets. Yet, they remain vulnerable to common data distortions such as adversarial and common corruption perturbations. These perturbations can significantly degrade performance, thereby challenging the overall reliability of the models. Traditional robustness validation typically relies on perturbed test datasets to assess and improve model performance. In our framework, however, we propose a validation approach that extracts "weak robust" samples directly from the training dataset via local robustness analysis. These samples, being the most susceptible to perturbations, serve as an early and sensitive indicator of the model's vulnerabilities. By evaluating models on these challenging training instances, we gain a more nuanced understanding of its robustness, which informs targeted performance enhancement. We demonstrate the effectiveness of our approach on models trained with CIFAR-10, CIFAR-100, and ImageNet, highlighting how robustness validation guided by weak robust samples can drive meaningful improvements in model reliability under adversarial and common corruption scenarios.