A Method of Generating Random Weights and Biases in Feedforward Neural Networks with Random Hidden Nodes
This work addresses a specific bottleneck in random neural networks for researchers and practitioners, offering an incremental improvement in model generalization.
The authors tackled the problem of selecting intervals for random weights and biases in neural networks with random hidden nodes, proposing a method that improves approximation performance by aligning activation functions with input data regions.
Neural networks with random hidden nodes have gained increasing interest from researchers and practical applications. This is due to their unique features such as very fast training and universal approximation property. In these networks the weights and biases of hidden nodes determining the nonlinear feature mapping are set randomly and are not learned. Appropriate selection of the intervals from which weights and biases are selected is extremely important. This topic has not yet been sufficiently explored in the literature. In this work a method of generating random weights and biases is proposed. This method generates the parameters of the hidden nodes in such a way that nonlinear fragments of the activation functions are located in the input space regions with data and can be used to construct the surface approximating a nonlinear target function. The weights and biases are dependent on the input data range and activation function type. The proposed methods allows us to control the generalization degree of the model. These all lead to improvement in approximation performance of the network. Several experiments show very promising results.