Revisiting Zeroth-Order Hessian Approximation: A Single-Step Policy Optimization Lens
This work provides a unified theoretical framework and practical estimators for improving derivative-free optimization methods, which is crucial for researchers and practitioners working on bilevel optimization, Bayesian inference, and uncertainty quantification.
This paper tackles the challenge of accurate Zeroth-Order (ZO) Hessian estimation in high-dimensional settings by reinterpreting it through single-step Policy Optimization. The proposed ZoVH framework introduces a suite of variance-reduced estimators for the full Hessian, its regularized inverse, and the bias-corrected inverse Hessian-gradient product, achieving superior estimation accuracy and convergence performance.
Accurate Zeroth-Order (ZO) Hessian estimation is a cornerstone of derivative-free methods, essential for tasks such as bilevel optimization, Bayesian inference, and uncertainty quantification. However, obtaining a complete suite of low-variance estimators for the Hessian and its inverse in high-dimensional settings remains a significant challenge. To address this, we propose a unified framework that reinterprets ZO Hessian approximation through the lens of single-step Policy Optimization (PO). This perspective establishes a theoretical equivalence between general ZO Hessian estimators and the Hessian of a smoothed PO objective, unifying distinct classical randomized estimators as specific instances of baseline selection. Building on this foundation, we introduce ZoVH, a comprehensive suite of variance-reduced estimators for the full Hessian matrix, its regularized inverse, and the bias-corrected inverse Hessian-gradient product. ZoVH leverages two key techniques: (1) a unique optimal baseline derived to provably minimize variance, and (2) a query reuse strategy that incorporates historical function queries to enhance sample efficiency without inflating costs. Our rigorous theoretical analysis confirms the unbiasedness of the Hessian estimator, validates the variance optimality of our baseline, provides error bounds for the entire ZoVH suite, and establishes convergence guarantees for the resulting curvature-aware ZO algorithm. Extensive empirical results validate our theoretical findings, demonstrating that ZoVH achieves superior estimation accuracy and convergence performance in real-world applications. Code is available at https://github.com/Qjbtiger/ZoVH