Optimization for Classical Machine Learning Problems on the GPU
This work addresses the problem of slow constrained optimization for machine learning practitioners by providing faster GPU-based solutions, though it is incremental as it builds on an existing framework.
The authors tackled the limited GPU support for constrained optimization in classical machine learning by extending the GENO framework to solve such problems on GPUs, resulting in solvers that outperform state-of-the-art approaches like CVXPY with GPU-accelerated solvers by a few orders of magnitude.
Constrained optimization problems arise frequently in classical machine learning. There exist frameworks addressing constrained optimization, for instance, CVXPY and GENO. However, in contrast to deep learning frameworks, GPU support is limited. Here, we extend the GENO framework to also solve constrained optimization problems on the GPU. The framework allows the user to specify constrained optimization problems in an easy-to-read modeling language. A solver is then automatically generated from this specification. When run on the GPU, the solver outperforms state-of-the-art approaches like CVXPY combined with a GPU-accelerated solver such as cuOSQP or SCS by a few orders of magnitude.