Stepsize Hedging: an Alternative Mechanism for Accelerating Gradient Descent

This work addresses the problem of accelerating gradient descent for machine learning practitioners by proposing an alternative mechanism for stepsize selection.

This article explores stepsize hedging as a mechanism to accelerate gradient descent by optimizing stepsize selection. It demonstrates that improved stepsize choices alone can lead to faster convergence.