Stochastic interior-point methods for smooth conic optimization with applications
This work addresses a bottleneck in applying conic optimization to large-scale machine learning problems, offering a more general and efficient solution for practitioners.
The authors tackled the lack of practical stochastic algorithms for general conic optimization in machine learning by introducing a stochastic interior-point method framework with four novel variants, achieving iteration complexity that matches the best-known results in stochastic unconstrained optimization up to a polylogarithmic factor.
Conic optimization plays a crucial role in many machine learning (ML) problems. However, practical algorithms for conic constrained ML problems with large datasets are often limited to specific use cases, as stochastic algorithms for general conic optimization remain underdeveloped. To fill this gap, we introduce a stochastic interior-point method (SIPM) framework for general conic optimization, along with four novel SIPM variants leveraging distinct stochastic gradient estimators. Under mild assumptions, we establish the iteration complexity of our proposed SIPMs, which, up to a polylogarithmic factor, match the best-known {results} in stochastic unconstrained optimization. Finally, our numerical experiments on robust linear regression, multi-task relationship learning, and clustering data streams demonstrate the effectiveness and efficiency of our approach.