Projection-Free Algorithm for Stochastic Bi-level Optimization
This addresses optimization challenges in machine learning, such as matrix completion and reinforcement learning, by providing efficient streaming solutions without projections, though it is incremental relative to single-level methods.
The paper tackles stochastic bi-level optimization problems by introducing the first projection-free algorithm, SBFW, achieving sample complexities of O(ε^{-3}) for convex and O(ε^{-4}) for non-convex objectives, with improved rates for a special case using SCFW.
This work presents the first projection-free algorithm to solve stochastic bi-level optimization problems, where the objective function depends on the solution of another stochastic optimization problem. The proposed $\textbf{S}$tochastic $\textbf{Bi}$-level $\textbf{F}$rank-$\textbf{W}$olfe ($\textbf{SBFW}$) algorithm can be applied to streaming settings and does not make use of large batches or checkpoints. The sample complexity of SBFW is shown to be $\mathcal{O}(ε^{-3})$ for convex objectives and $\mathcal{O}(ε^{-4})$ for non-convex objectives. Improved rates are derived for the stochastic compositional problem, which is a special case of the bi-level problem, and entails minimizing the composition of two expected-value functions. The proposed $\textbf{S}$tochastic $\textbf{C}$ompositional $\textbf{F}$rank-$\textbf{W}$olfe ($\textbf{SCFW}$) is shown to achieve a sample complexity of $\mathcal{O}(ε^{-2})$ for convex objectives and $\mathcal{O}(ε^{-3})$ for non-convex objectives, at par with the state-of-the-art sample complexities for projection-free algorithms solving single-level problems. We demonstrate the advantage of the proposed methods by solving the problem of matrix completion with denoising and the problem of policy value evaluation in reinforcement learning.