Multi-Activation Hidden Units for Neural Networks with Random Weights
This addresses a computational efficiency issue for users of random-weight neural networks, though it appears incremental as it builds on existing methods.
The paper tackles the problem of single-layer feedforward networks with random weights requiring many hidden units by proposing multi-activation hidden units, which improve classification accuracy or reduce computations without increasing hidden unit count.
Single layer feedforward networks with random weights are successful in a variety of classification and regression problems. These networks are known for their non-iterative and fast training algorithms. A major drawback of these networks is that they require a large number of hidden units. In this paper, we propose the use of multi-activation hidden units. Such units increase the number of tunable parameters and enable formation of complex decision surfaces, without increasing the number of hidden units. We experimentally show that multi-activation hidden units can be used either to improve the classification accuracy, or to reduce computations.