Remarks on Lipschitz-Minimal Interpolation: Generalization Bounds and Neural Network Implementation
This provides a theoretical and practical framework for certified generalization in machine learning, particularly for safety-critical applications like control systems, though it is incremental in extending Lipschitz-based methods to neural networks.
The paper tackles the problem of approximating functions with guaranteed generalization error bounds by selecting interpolating functions with minimal Lipschitz constants, establishing rigorous bounds for classes like deep neural networks and implementing a neural network method with certified error bounds for learning system dynamics.
This note establishes a theoretical framework for finding (potentially overparameterized) approximations of a function on a compact set with a-priori bounds for the generalization error. The approximation method considered is to choose, among all functions that (approximately) interpolate a given data set, one with a minimal Lipschitz constant. The paper establishes rigorous generalization bounds over practically relevant classes of approximators, including deep neural networks. It also presents a neural network implementation based on Lipschitz-bounded network layers and an augmented Lagrangian method. The results are illustrated for a problem of learning the dynamics of an input-to-state stable system with certified bounds on simulation error.