Stochastic Zeroth-Order Method for Computing Generalized Rayleigh Quotients
It provides a novel method for numerical linear algebra problems where adjoint mismatches are problematic, but the improvement over existing methods is incremental.
The paper introduces a stochastic zeroth-order Riemannian algorithm for maximizing generalized Rayleigh quotients without requiring adjoint or matrix inverse computations, and proves sublinear convergence to global maximizers with probability one.
The maximization of the (generalized) Rayleigh quotient is a central problem in numerical linear algebra. Conventional algorithms for its computation typically rely on matrix-adjoint products, making them sensitive to errors arrissing from adjoint mismatches. To address this issue, we introduce a stochastic zeroth-order Riemannian algorithm that maximizes the generalized Rayleigh quotient without requiring adjoint or matrix inverse computations. We provide theoretical convergence guarantees showing that the iterates converge to the set of global maximizers of the (generalized) Rayleigh quotient and the norm of the Riemannian gradient vanishes at a sublinear rate with probability one. Our theoretical results are supported by numerical experiments, which demonstrate the excellent performance of the proposed method compared to state-of-the-art algorithms.