Neural Fixed-Point Acceleration for Convex Optimization
This work addresses the need for faster optimization in real-time applications, but it is incremental as it builds on existing acceleration methods and applies them to a specific solver.
The paper tackles the computational bottleneck of fixed-point iterations in real-time applications by introducing neural fixed-point acceleration, which learns to accelerate problems from a distribution, and applies it to SCS, the state-of-the-art solver for convex cone programming, achieving unspecified speed improvements.
Fixed-point iterations are at the heart of numerical computing and are often a computational bottleneck in real-time applications that typically need a fast solution of moderate accuracy. We present neural fixed-point acceleration which combines ideas from meta-learning and classical acceleration methods to automatically learn to accelerate fixed-point problems that are drawn from a distribution. We apply our framework to SCS, the state-of-the-art solver for convex cone programming, and design models and loss functions to overcome the challenges of learning over unrolled optimization and acceleration instabilities. Our work brings neural acceleration into any optimization problem expressible with CVXPY. The source code behind this paper is available at https://github.com/facebookresearch/neural-scs