Quasi-Quadratic Gradient: A New Direction for Accelerating the BFGS Method in Quasi-Newton Optimization
For optimization practitioners, this offers a simple modification to improve BFGS convergence speed without added computational cost.
The paper introduces the Quasi-Quadratic Gradient (QQG) to accelerate the BFGS method, achieving faster convergence than vanilla BFGS while maintaining computational efficiency.
In this paper, we introduce the Quasi-Quadratic Gradient (QQG), a novel search direction designed to accelerate the BFGS method within the quasi-Newton framework. By defining the QQG as the product of the inverse Hessian approximation and the current gradient, we explicitly leverage local second-order curvature to rectify the search path. Theoretical analysis and empirical results demonstrate that our approach significantly outperforms vanilla BFGS in convergence speed while maintaining computational efficiency.