Accelerating Adversarial Perturbation by 50% with Semi-backward Propagation
This work addresses efficiency improvements for adversarial robustness research, though it appears incremental as it builds on existing optimization techniques.
The paper tackles the problem of accelerating adversarial perturbation optimization by proposing a method that avoids computing parameter gradients during backward propagation, achieving a 50% overall speedup and a 2x acceleration in backward propagation with no utility drop.
Adversarial perturbation plays a significant role in the field of adversarial robustness, which solves a maximization problem over the input data. We show that the backward propagation of such optimization can accelerate $2\times$ (and thus the overall optimization including the forward propagation can accelerate $1.5\times$), without any utility drop, if we only compute the output gradient but not the parameter gradient during the backward propagation.