Improving Randomized Learning of Feedforward Neural Networks by Appropriate Generation of Random Parameters
This is an incremental improvement for researchers and practitioners in randomized neural network learning, focusing on parameter initialization.
The authors tackled the problem of generating random parameters for single-hidden-layer feedforward neural networks by proposing a method that adjusts slope angles and rotations based on the target function, resulting in better performance for complex functions compared to fixed-interval approaches.
In this work, a method of random parameters generation for randomized learning of a single-hidden-layer feedforward neural network is proposed. The method firstly, randomly selects the slope angles of the hidden neurons activation functions from an interval adjusted to the target function, then randomly rotates the activation functions, and finally distributes them across the input space. For complex target functions the proposed method gives better results than the approach commonly used in practice, where the random parameters are selected from the fixed interval. This is because it introduces the steepest fragments of the activation functions into the input hypercube, avoiding their saturation fragments.