Shallow ReLU neural networks and finite elements
This bridges shallow ReLU neural networks and finite element methods, offering a new perspective for approximation analysis, but it is incremental as it builds on existing mesh and network concepts.
The paper shows that piecewise linear functions on a convex polytope mesh can be weakly represented by two-hidden-layer ReLU neural networks, with neuron counts based on mesh properties, and extends this to finite element functions to analyze approximation capabilities in L^p norm.
We point out that (continuous or discontinuous) piecewise linear functions on a convex polytope mesh can be represented by two-hidden-layer ReLU neural networks in a weak sense. In addition, the numbers of neurons of the two hidden layers required to weakly represent are accurately given based on the numbers of polytopes and hyperplanes involved in this mesh. The results naturally hold for constant and linear finite element functions. Such weak representation establishes a bridge between shallow ReLU neural networks and finite element functions, and leads to a perspective for analyzing approximation capability of ReLU neural networks in $L^p$ norm via finite element functions. Moreover, we discuss the strict representation for tensor finite element functions via the recent tensor neural networks.