Chebyshev Feature Neural Network for Accurate Function Approximation
This addresses the challenge of accurate function approximation in machine learning, offering a domain-specific improvement for tasks requiring high precision.
The paper tackled the problem of approximating functions with high accuracy by introducing the Chebyshev Feature Neural Network (CFNN), which achieved machine accuracy in training through a novel architecture with learnable Chebyshev frequencies and a multi-stage strategy, as demonstrated in numerical examples up to 20 dimensions.
We present a new Deep Neural Network (DNN) architecture capable of approximating functions up to machine accuracy. Termed Chebyshev Feature Neural Network (CFNN), the new structure employs Chebyshev functions with learnable frequencies as the first hidden layer, followed by the standard fully connected hidden layers. The learnable frequencies of the Chebyshev layer are initialized with exponential distributions to cover a wide range of frequencies. Combined with a multi-stage training strategy, we demonstrate that this CFNN structure can achieve machine accuracy during training. A comprehensive set of numerical examples for dimensions up to $20$ are provided to demonstrate the effectiveness and scalability of the method.