Estimating the Jacobian matrix of an unknown multivariate function from sample values by means of a neural network
This method provides structural information about functions, moving beyond black-box neural network approximations, which is useful for applications in science and engineering where Jacobian matrices are critical.
The authors tackled the problem of estimating the Jacobian matrix of an unknown multivariate function using only sample values, without explicit derivative information, by training a neural network with a loss function based on linear approximations and nearest neighbor search. They achieved a formal upper bound on the error between the estimated and actual Jacobian matrix under specified assumptions.
We describe, implement and test a novel method for training neural networks to estimate the Jacobian matrix $J$ of an unknown multivariate function $F$. The training set is constructed from finitely many pairs $(x,F(x))$ and it contains no explicit information about $J$. The loss function for backpropagation is based on linear approximations and on a nearest neighbor search in the sample data. We formally establish an upper bound on the uniform norm of the error, in operator norm, between the estimated Jacobian matrix provided by the algorithm and the actual Jacobian matrix, under natural assumptions on the function, on the training set and on the loss of the neural network during training. The Jacobian matrix of a multivariate function contains a wealth of information about the function and it has numerous applications in science and engineering. The method given here represents a step in moving from black-box approximations of functions by neural networks to approximations that provide some structural information about the function in question.