Stochastic Two Points Method for Deep Model Zeroth-order Optimization
This addresses the problem of hardware and access limitations for training large AI models, offering a practical zeroth-order optimization solution, though it appears incremental as it builds on existing gradient-free methods.
The paper tackles the challenge of optimizing large foundation models without backpropagation by introducing an efficient Stochastic Two-Point (S2P) method, showing that its variant VS2P outperforms or matches standard methods across various model types and scales in empirical tests.
Large foundation models, such as large language models, have performed exceptionally well in various application scenarios. Building or fully fine-tuning such large models is usually prohibitive due to either hardware budget or lack of access to backpropagation. The zeroth-order methods offer a promising direction for tackling this challenge, where only forward passes are needed to update the model. This paper introduces an efficient Stochastic Two-Point (S2P) approach within the gradient-free regime. We present the theoretical convergence properties of S2P under the general and relaxed smoothness assumptions, and the derived results help understand and inherently connect the two popular types of zeroth-order methods, basic random search and stochastic three-point method. The theoretical properties also shed light on a Variant of S2P (VS2P), through exploiting our new convergence properties that better represent the dynamics of deep models in training. Our comprehensive empirical results show that VS2P is highly effective in optimizing objectives for deep models. It outperforms or achieves competitive performance compared to standard methods across various model types and scales.