Accelerating Legacy Numerical Solvers by Non-intrusive Gradient-based Meta-solving
This work addresses a bottleneck for practitioners in scientific computing who rely on established but non-differentiable legacy codes, enabling them to apply meta-learning techniques for hyperparameter selection.
The paper tackles the problem of accelerating legacy numerical solvers that are not automatically differentiable by proposing a non-intrusive gradient-based meta-solving method, achieving theoretical and numerical advantages over baselines without modifying existing codes.
Scientific computing is an essential tool for scientific discovery and engineering design, and its computational cost is always a main concern in practice. To accelerate scientific computing, it is a promising approach to use machine learning (especially meta-learning) techniques for selecting hyperparameters of traditional numerical methods. There have been numerous proposals to this direction, but many of them require automatic-differentiable numerical methods. However, in reality, many practical applications still depend on well-established but non-automatic-differentiable legacy codes, which prevents practitioners from applying the state-of-the-art research to their own problems. To resolve this problem, we propose a non-intrusive methodology with a novel gradient estimation technique to combine machine learning and legacy numerical codes without any modification. We theoretically and numerically show the advantage of the proposed method over other baselines and present applications of accelerating established non-automatic-differentiable numerical solvers implemented in PETSc, a widely used open-source numerical software library.