Prevention of Overfitting on Mesh-Structured Data Regressions with a Modified Laplace Operator
This is an incremental improvement for researchers working with mesh-structured data in regression tasks, as it offers a method to prevent overfitting without traditional testing splits.
The paper tackles overfitting in mesh-structured data regressions by using a modified Laplace operator to compute derivatives as a true label for data entropy and as a surrogate testing metric, achieving a reduction in unwanted oscillations through hyperparameter optimization without requiring data splitting.
This document reports on a method for detecting and preventing overfitting on data regressions, herein applied to mesh-like data structures. The mesh structure allows for the straightforward computation of the Laplace-operator second-order derivatives in a finite-difference fashion for noiseless data. Derivatives of the training data are computed on the original training mesh to serve as a true label of the entropy of the training data. Derivatives of the trained data are computed on a staggered mesh to identify oscillations in the interior of the original training mesh cells. The loss of the Laplace-operator derivatives is used for hyperparameter optimisation, achieving a reduction of unwanted oscillation through the minimisation of the entropy of the trained model. In this setup, testing does not require the splitting of points from the training data, and training is thus directly performed on all available training points. The Laplace operator applied to the trained data on a staggered mesh serves as a surrogate testing metric based on diffusion properties.