Deep Stochastic Configuration Networks with Universal Approximation Property
This work addresses the challenge of efficient deep network construction for researchers and practitioners in machine learning, though it appears incremental as it builds on existing randomized methods.
The paper tackles the problem of building deep neural networks efficiently by introducing a randomized approach with a supervisory mechanism for weight assignment and direct links from all hidden layers to the output, establishing universal approximation property and demonstrating speed in learning representations.
This paper develops a randomized approach for incrementally building deep neural networks, where a supervisory mechanism is proposed to constrain the random assignment of the weights and biases, and all the hidden layers have direct links to the output layer. A fundamental result on the universal approximation property is established for such a class of randomized leaner models, namely deep stochastic configuration networks (DeepSCNs). A learning algorithm is presented to implement DeepSCNs with either specific architecture or self-organization. The read-out weights attached with all direct links from each hidden layer to the output layer are evaluated by the least squares method. Given a set of training examples, DeepSCNs can speedily produce a learning representation, that is, a collection of random basis functions with the cascaded inputs together with the read-out weights. An empirical study on a function approximation is carried out to demonstrate some properties of the proposed deep learner model.