Limited-memory BFGS Systems with Diagonal Updates
This work provides a computationally efficient solution for a bottleneck in large-scale optimization methods like trust-region and doubly-augmented Lagrangian methods.
The authors propose a recursion formula to solve linear systems involving a limited-memory BFGS matrix plus a scaled identity, achieving O(M^2 n) complexity. The method is shown to be efficient for large-scale optimization problems.
In this paper, we investigate a formula to solve systems of the form (B + σI)x = y, where B is a limited-memory BFGS quasi-Newton matrix and σ is a positive constant. These types of systems arise naturally in large-scale optimization such as trust-region methods as well as doubly-augmented Lagrangian methods. We show that provided a simple condition holds on B_0 and σ, the system (B + σI)x = y can be solved via a recursion formula that requies only vector inner products. This formula has complexity M^2n, where M is the number of L-BFGS updates and n >> M is the dimension of x.