Riemannian Stochastic Gradient Method for Nested Composition Optimization
This addresses a bottleneck in applications like reinforcement learning policy evaluation and meta-learning model customization, offering a novel method for a specific problem.
The paper tackles optimization of nested composition functions over Riemannian manifolds, where stochastic approximations cause bias in gradients, and presents a Riemannian Stochastic Composition Gradient Descent (R-SCGD) method that achieves an expected squared Riemannian gradient smaller than ε in O(ε^{-2}) oracle calls.
This work considers optimization of composition of functions in a nested form over Riemannian manifolds where each function contains an expectation. This type of problems is gaining popularity in applications such as policy evaluation in reinforcement learning or model customization in meta-learning. The standard Riemannian stochastic gradient methods for non-compositional optimization cannot be directly applied as stochastic approximation of inner functions create bias in the gradients of the outer functions. For two-level composition optimization, we present a Riemannian Stochastic Composition Gradient Descent (R-SCGD) method that finds an approximate stationary point, with expected squared Riemannian gradient smaller than $ε$, in $O(ε^{-2})$ calls to the stochastic gradient oracle of the outer function and stochastic function and gradient oracles of the inner function. Furthermore, we generalize the R-SCGD algorithms for problems with multi-level nested compositional structures, with the same complexity of $O(ε^{-2})$ for the first-order stochastic oracle. Finally, the performance of the R-SCGD method is numerically evaluated over a policy evaluation problem in reinforcement learning.