An Empirical Evaluation of Adversarial Robustness under Transfer Learning
This work addresses the problem of maintaining adversarial defenses in transfer learning for machine learning practitioners, though it is incremental as it builds on existing methods.
The paper investigates how adversarial robustness transfers from a source model trained on CIFAR-100 to a target model on CIFAR-10, finding that using PGD examples in source training yields more general robust features and improves accuracy by 5.2% against white-box PGD attacks.
In this work, we evaluate adversarial robustness in the context of transfer learning from a source trained on CIFAR 100 to a target network trained on CIFAR 10. Specifically, we study the effects of using robust optimisation in the source and target networks. This allows us to identify transfer learning strategies under which adversarial defences are successfully retained, in addition to revealing potential vulnerabilities. We study the extent to which features learnt by a fast gradient sign method (FGSM) and its iterative alternative (PGD) can preserve their defence properties against black and white-box attacks under three different transfer learning strategies. We find that using PGD examples during training on the source task leads to more general robust features that are easier to transfer. Furthermore, under successful transfer, it achieves 5.2% more accuracy against white-box PGD attacks than suitable baselines. Overall, our empirical evaluations give insights on how well adversarial robustness under transfer learning can generalise.