About rectified sigmoid function for enhancing the accuracy of Physics-Informed Neural Networks
This work addresses accuracy issues in physics-informed neural networks for solving ODEs, but it is incremental as it modifies an existing activation function.
The authors tackled the problem of solving physical ODEs with neural networks by proposing a rectified sigmoid activation function, which improved accuracy over standard sigmoid networks in numerical experiments on systems like the harmonic oscillator and Lorentz system.
The article is devoted to the study of neural networks with one hidden layer and a modified activation function for solving physical problems. A rectified sigmoid activation function has been proposed to solve physical problems described by the ODE with neural networks. Algorithms for physics-informed data-driven initialization of a neural network and a neuron-by-neuron gradient-free fitting method have been presented for the neural network with this activation function. Numerical experiments demonstrate the superiority of neural networks with a rectified sigmoid function over neural networks with a sigmoid function in the accuracy of solving physical problems (harmonic oscillator, relativistic slingshot, and Lorentz system).