Mixture of Robust Experts (MoRE):A Robust Denoising Method towards multiple perturbations
This work addresses the need for robust AI models that can handle diverse adversarial and environmental perturbations, representing an incremental improvement over existing adversarial training methods.
The paper tackles the problem of deep neural networks' susceptibility to adversarial examples and limited flexibility in handling multiple perturbation types by proposing Mixture of Robust Experts (MoRE), which uses a gating mechanism to dynamically combine expert networks trained for specific perturbations or clean data, achieving superior accuracy across various data types.
To tackle the susceptibility of deep neural networks to adversarial examples, the adversarial training has been proposed which provides a notion of security through an inner maximization problem presenting the first-order adversaries embedded within the outer minimization of the training loss. To generalize the adversarial robustness over different perturbation types, the adversarial training method has been augmented with the improved inner maximization presenting a union of multiple perturbations e.g., various $\ell_p$ norm-bounded perturbations. However, the improved inner maximization only enjoys limited flexibility in terms of the allowable perturbation types. In this work, through a gating mechanism, we assemble a set of expert networks, each one either adversarially trained to deal with a particular perturbation type or normally trained for boosting accuracy on clean data. The gating module assigns weights dynamically to each expert to achieve superior accuracy under various data types e.g., adversarial examples, adverse weather perturbations, and clean input. In order to deal with the obfuscated gradients issue, the training of the gating module is conducted together with fine-tuning of the last fully connected layers of expert networks through adversarial training approach. Using extensive experiments, we show that our Mixture of Robust Experts (MoRE) approach enables a flexible integration of a broad range of robust experts with superior performance.