HO-FMN: Hyperparameter Optimization for Fast Minimum-Norm Attacks
This work addresses the need for more accurate adversarial robustness evaluation in machine learning, particularly for researchers and practitioners assessing model security, though it is incremental as it builds on an existing attack method.
The paper tackled the problem of overly-optimistic robustness evaluations in gradient-based attacks by proposing a parametric variation of the fast minimum-norm attack algorithm, which dynamically adjusts loss, optimizer, scheduler, and hyperparameters, resulting in finding smaller adversarial perturbations without additional tuning and enabling more complete robustness reporting.
Gradient-based attacks are a primary tool to evaluate robustness of machine-learning models. However, many attacks tend to provide overly-optimistic evaluations as they use fixed loss functions, optimizers, step-size schedulers, and default hyperparameters. In this work, we tackle these limitations by proposing a parametric variation of the well-known fast minimum-norm attack algorithm, whose loss, optimizer, step-size scheduler, and hyperparameters can be dynamically adjusted. We re-evaluate 12 robust models, showing that our attack finds smaller adversarial perturbations without requiring any additional tuning. This also enables reporting adversarial robustness as a function of the perturbation budget, providing a more complete evaluation than that offered by fixed-budget attacks, while remaining efficient. We release our open-source code at https://github.com/pralab/HO-FMN.