A Zeroth-Order Block Coordinate Descent Algorithm for Huge-Scale Black-Box Optimization
This addresses the challenge of black-box optimization for high-dimensional problems, such as adversarial attacks in machine learning, with incremental improvements in efficiency.
The paper tackles the problem of zeroth-order optimization in huge-scale settings where high dimensions make vector operations infeasible, proposing the ZO-BCD algorithm that reduces per-iteration complexity and achieves a 97.9% attack success rate in adversarial attacks on audio classifiers.
We consider the zeroth-order optimization problem in the huge-scale setting, where the dimension of the problem is so large that performing even basic vector operations on the decision variables is infeasible. In this paper, we propose a novel algorithm, coined ZO-BCD, that exhibits favorable overall query complexity and has a much smaller per-iteration computational complexity. In addition, we discuss how the memory footprint of ZO-BCD can be reduced even further by the clever use of circulant measurement matrices. As an application of our new method, we propose the idea of crafting adversarial attacks on neural network based classifiers in a wavelet domain, which can result in problem dimensions of over 1.7 million. In particular, we show that crafting adversarial examples to audio classifiers in a wavelet domain can achieve the state-of-the-art attack success rate of 97.9%.