Worst-case Nonlinear Regression with Error Bounds
For practitioners needing reliable surrogate models with guaranteed error bounds in safety-critical applications, this work provides a practical active-learning approach with theoretical guarantees.
This paper introduces an active-learning method for nonlinear minimax regression that fits a surrogate model by minimizing the maximum absolute approximation error, using a smooth L∞ approximation for efficient gradient-based training. The method achieves worst-case error bounds and is validated on several benchmarks, including nonlinear function approximation and explicit model predictive control.
We propose an active-learning method for nonlinear minimax regression. Given a nonlinear function that can be arbitrarily evaluated over a compact set, we fit a surrogate model, such as a feedforward neural network, by minimizing the maximum absolute approximation error. To handle the nonsmoothness of this worst-case loss, we introduce a smooth $L_\infty$ approximation that enables efficient gradient-based training. The training set is iteratively enriched by querying points of largest error via global optimization. We also derive constant and input-dependent worst-case error bounds over the entire input domain. The approach is validated on approximations of nonlinear functions and nonconvex sets, uncertain models of nonlinear dynamics, and explicit model predictive control laws. A Python library is available at https://github.com/bemporad/maxfit.