Information Preserving Line Search via Bayesian Optimization
This work addresses a fundamental bottleneck in iterative optimization for researchers and practitioners, offering an incremental improvement over existing methods.
The paper tackles the inefficiency of traditional line search methods in optimization by proposing a Bayesian optimization approach that preserves and utilizes previously discarded function and gradient information, resulting in superior performance on the CUTEst test set.
Line search is a fundamental part of iterative optimization methods for unconstrained and bound-constrained optimization problems to determine suitable step lengths that provide sufficient improvement in each iteration. Traditional line search methods are based on iterative interval refinement, where valuable information about function value and gradient is discarded in each iteration. We propose a line search method via Bayesian optimization, preserving and utilizing otherwise discarded information to improve step-length choices. Our approach is guaranteed to converge and shows superior performance compared to state-of-the-art methods based on empirical tests on the challenging unconstrained and bound-constrained optimization problems from the CUTEst test set.