Learning Gradient Flow: Using Equation Discovery to Accelerate Engineering Optimization
This work addresses computational bottlenecks in engineering optimization for researchers and practitioners, offering a novel method to reduce evaluation costs, though it is incremental as it builds on existing optimization techniques.
The paper tackles the problem of expensive objective and gradient evaluations in optimization by using data-driven equation discovery to learn continuous-time dynamics from trajectory data, creating the Learned Gradient Flow (LGF) optimizer as a surrogate. Results show that this approach significantly expedites convergence on engineering and scientific machine learning problems, such as inverse problems and structural topology optimization.
In this work, we investigate the use of data-driven equation discovery for dynamical systems to model and forecast continuous-time dynamics of unconstrained optimization problems. To avoid expensive evaluations of the objective function and its gradient, we leverage trajectory data on the optimization variables to learn the continuous-time dynamics associated with gradient descent, Newton's method, and ADAM optimization. The discovered gradient flows are then solved as a surrogate for the original optimization problem. To this end, we introduce the Learned Gradient Flow (LGF) optimizer, which is equipped to build surrogate models of variable polynomial order in full- or reduced-dimensional spaces at user-defined intervals in the optimization process. We demonstrate the efficacy of this approach on several standard problems from engineering mechanics and scientific machine learning, including two inverse problems, structural topology optimization, and two forward solves with different discretizations. Our results suggest that the learned gradient flows can significantly expedite convergence by capturing critical features of the optimization trajectory while avoiding expensive evaluations of the objective and its gradient.