Boosting One-Point Derivative-Free Online Optimization via Residual Feedback
This work addresses a bottleneck in zeroth-order optimization for online settings, offering improved performance for applications like real-time control or adaptive systems, though it is incremental as it builds on existing one-point feedback methods.
The paper tackles the problem of online optimization with time-varying objective functions where only one function value query is possible per time step, proposing a one-point feedback method using residual feedback that yields tighter regret bounds and smaller gradient estimate variance compared to conventional methods.
Zeroth-order optimization (ZO) typically relies on two-point feedback to estimate the unknown gradient of the objective function. Nevertheless, two-point feedback can not be used for online optimization of time-varying objective functions, where only a single query of the function value is possible at each time step. In this work, we propose a new one-point feedback method for online optimization that estimates the objective function gradient using the residual between two feedback points at consecutive time instants. Moreover, we develop regret bounds for ZO with residual feedback for both convex and nonconvex online optimization problems. Specifically, for both deterministic and stochastic problems and for both Lipschitz and smooth objective functions, we show that using residual feedback can produce gradient estimates with much smaller variance compared to conventional one-point feedback methods. As a result, our regret bounds are much tighter compared to existing regret bounds for ZO with conventional one-point feedback, which suggests that ZO with residual feedback can better track the optimizer of online optimization problems. Additionally, our regret bounds rely on weaker assumptions than those used in conventional one-point feedback methods. Numerical experiments show that ZO with residual feedback significantly outperforms existing one-point feedback methods also in practice.