On Sharp Stochastic Zeroth Order Hessian Estimators over Riemannian Manifolds
This work addresses a foundational challenge in optimization over curved spaces, providing the first geometry-dependent bias bound for Hessian estimators, which is incremental but important for applications in machine learning and physics.
The paper tackles the problem of estimating Hessians for functions on Riemannian manifolds with limited function evaluations, introducing a stochastic zeroth-order estimator that achieves a bias bound of O(γδ²), where γ depends on manifold geometry and function properties, and demonstrates empirical superiority.
We study Hessian estimators for functions defined over an $n$-dimensional complete analytic Riemannian manifold. We introduce new stochastic zeroth-order Hessian estimators using $O (1)$ function evaluations. We show that, for an analytic real-valued function $f$, our estimator achieves a bias bound of order $ O \left( γδ^2 \right) $, where $ γ$ depends on both the Levi-Civita connection and function $f$, and $δ$ is the finite difference step size. To the best of our knowledge, our results provide the first bias bound for Hessian estimators that explicitly depends on the geometry of the underlying Riemannian manifold. We also study downstream computations based on our Hessian estimators. The supremacy of our method is evidenced by empirical evaluations.