Overestimation learning with guarantees
This method addresses the need for reliable surrogate functions in critical applications where underestimation is unacceptable, such as embedded systems with limited resources, though it is incremental as it builds on existing overestimation techniques with a monotonicity assumption.
The paper tackles the problem of learning a neural network that is guaranteed to overestimate a monotonic reference function on a given domain, enabling its use as a surrogate in resource-constrained systems like embedded devices. Experiments on synthetic and real data show that the learned over-estimations provide good approximations while maintaining the guarantee.
We describe a complete method that learns a neural network which is guaranteed to overestimate a reference function on a given domain. The neural network can then be used as a surrogate for the reference function. The method involves two steps. In the first step, we construct an adaptive set of Majoring Points. In the second step, we optimize a well-chosen neural network to overestimate the Majoring Points. In order to extend the guarantee on the Majoring Points to the whole domain, we necessarily have to make an assumption on the reference function. In this study, we assume that the reference function is monotonic. We provide experiments on synthetic and real problems. The experiments show that the density of the Majoring Points concentrate where the reference function varies. The learned over-estimations are both guaranteed to overestimate the reference function and are proven empirically to provide good approximations of it. Experiments on real data show that the method makes it possible to use the surrogate function in embedded systems for which an underestimation is critical; when computing the reference function requires too many resources.