Weights initialization of neural networks for function approximation
This addresses efficiency and generalization issues in neural network-based function approximation for scientific computing and machine learning, though it is incremental as it builds on existing initialization strategies.
The paper tackles the challenges of training neural networks for function approximation, such as needing new models for each function and poor generalization, by proposing a reusable initialization framework based on pretraining basis functions, which leads to substantial improvements in training efficiency and generalization in numerical experiments.
Neural network-based function approximation plays a pivotal role in the advancement of scientific computing and machine learning. Yet, training such models faces several challenges: (i) each target function often requires training a new model from scratch; (ii) performance is highly sensitive to architectural and hyperparameter choices; and (iii) models frequently generalize poorly beyond the training domain. To overcome these challenges, we propose a reusable initialization framework based on basis function pretraining. In this approach, basis neural networks are first trained to approximate families of polynomials on a reference domain. Their learned parameters are then used to initialize networks for more complex target functions. To enhance adaptability across arbitrary domains, we further introduce a domain mapping mechanism that transforms inputs into the reference domain, thereby preserving structural correspondence with the pretrained models. Extensive numerical experiments in one- and two-dimensional settings demonstrate substantial improvements in training efficiency, generalization, and model transferability, highlighting the promise of initialization-based strategies for scalable and modular neural function approximation. The full code is made publicly available on Gitee.