Conjugate Gradients and Accelerated Methods Unified: The Approximate Duality Gap View
This work provides a unified theoretical framework for optimization methods, but it is incremental as it builds on existing techniques without introducing a new paradigm.
The paper presents a new, self-contained analysis of the conjugate gradient method for convex quadratic minimization, clarifying its relationship with accelerated methods through the Approximate Duality Gap Technique.
This note provides a novel, simple analysis of the method of conjugate gradients for the minimization of convex quadratic functions. In contrast with standard arguments, our proof is entirely self-contained and does not rely on the existence of Chebyshev polynomials. Another advantage of our development is that it clarifies the relation between the method of conjugate gradients and general accelerated methods for smooth minimization by unifying their analyses within the framework of the Approximate Duality Gap Technique that was introduced by the authors.