Conic Descent Redux for Memory-Efficient Optimization
This work addresses memory efficiency in optimization for signal processing and machine learning tasks, though it appears incremental as it builds on existing conic descent methods.
The paper revisits conic descent optimization to improve its intuition, theory, and implementation, resulting in a momentum variant (MOCO) and a memory-efficient version that scales semidefinite programming for low-rank solutions, with numerical validation.
Conic programming has well-documented merits in a gamut of signal processing and machine learning tasks. This contribution revisits a recently developed first-order conic descent (CD) solver, and advances it in three aspects: intuition, theory, and algorithmic implementation. It is found that CD can afford an intuitive geometric derivation that originates from the dual problem. This opens the door to novel algorithmic designs, with a momentum variant of CD, momentum conic descent (MOCO) exemplified. Diving deeper into the dual behavior CD and MOCO reveals: i) an analytically justified stopping criterion; and, ii) the potential to design preconditioners to speed up dual convergence. Lastly, to scale semidefinite programming (SDP) especially for low-rank solutions, a memory efficient MOCO variant is developed and numerically validated.