Information-Theoretic Trust Regions for Stochastic Gradient-Based Optimization
This addresses the problem of improving optimization stability and speed for neural network training, though it is incremental as it builds on existing trust region and adaptive methods.
The paper tackles the challenge of efficiently using second-order information in stochastic gradient optimization by proposing Information-Theoretic Trust Region Optimization (arTuRO), which models parameters as a Gaussian distribution and uses a KL divergence-based trust region to compute optimal step sizes, resulting in faster and more stable optimization that combines fast convergence with good generalization.
Stochastic gradient-based optimization is crucial to optimize neural networks. While popular approaches heuristically adapt the step size and direction by rescaling gradients, a more principled approach to improve optimizers requires second-order information. Such methods precondition the gradient using the objective's Hessian. Yet, computing the Hessian is usually expensive and effectively using second-order information in the stochastic gradient setting is non-trivial. We propose using Information-Theoretic Trust Region Optimization (arTuRO) for improved updates with uncertain second-order information. By modeling the network parameters as a Gaussian distribution and using a Kullback-Leibler divergence-based trust region, our approach takes bounded steps accounting for the objective's curvature and uncertainty in the parameters. Before each update, it solves the trust region problem for an optimal step size, resulting in a more stable and faster optimization process. We approximate the diagonal elements of the Hessian from stochastic gradients using a simple recursive least squares approach, constructing a model of the expected Hessian over time using only first-order information. We show that arTuRO combines the fast convergence of adaptive moment-based optimization with the generalization capabilities of SGD.