Reinforcement learning for online hyperparameter tuning in convex quadratic programming
This work addresses hyperparameter tuning for quadratic programming solvers, which is an incremental improvement in optimization methods for control and data science applications.
The paper tackled the problem of slow convergence and hyperparameter tuning in quadratic programming solvers by using reinforcement learning to tune parameters online, resulting in a learned policy that generalizes well across different problem classes and configurations after lightweight training.
Quadratic programming is a workhorse of modern nonlinear optimization, control, and data science. Although regularized methods offer convergence guarantees under minimal assumptions on the problem data, they can exhibit the slow tail-convergence typical of first-order schemes, thus requiring many iterations to achieve high-accuracy solutions. Moreover, hyperparameter tuning significantly impacts on the solver performance but how to find an appropriate parameter configuration remains an elusive research question. To address these issues, we explore how data-driven approaches can accelerate the solution process. Aiming at high-accuracy solutions, we focus on a stabilized interior-point solver and carefully handle its two-loop flow and control parameters. We will show that reinforcement learning can make a significant contribution to facilitating the solver tuning and to speeding up the optimization process. Numerical experiments demonstrate that, after a lightweight training, the learned policy generalizes well to different problem classes with varying dimensions and to various solver configurations.