Complexity of Linear Minimization and Projection on Some Sets
This work addresses a gap in optimization theory by providing a more complete analysis of computational advantages, which is incremental as it builds on existing discussions but offers new methods for specific sets.
The paper tackles the problem of comparing computational complexities of linear minimization and projection tasks in constrained optimization, specifically for the Frank-Wolfe algorithm, by reviewing complexity bounds on common sets and proposing new projection methods for the ℓp-ball and Birkhoff polytope.
The Frank-Wolfe algorithm is a method for constrained optimization that relies on linear minimizations, as opposed to projections. Therefore, a motivation put forward in a large body of work on the Frank-Wolfe algorithm is the computational advantage of solving linear minimizations instead of projections. However, the discussions supporting this advantage are often too succinct or incomplete. In this paper, we review the complexity bounds for both tasks on several sets commonly used in optimization. Projection methods onto the $\ell_p$-ball, $p\in\left]1,2\right[\cup\left]2,+\infty\right[$, and the Birkhoff polytope are also proposed.