Efficient Decremental Learning Algorithms for Broad Learning System
This work addresses efficiency in machine learning for systems requiring model reduction, but it is incremental as it builds directly on prior incremental algorithms.
The paper tackles the problem of decremental learning in Broad Learning Systems by proposing algorithms to prune redundant nodes and remove obsolete training samples, deducing them from existing incremental learning methods to update output weights recursively.
The decremented learning algorithms are required in machine learning, to prune redundant nodes and remove obsolete inline training samples. In this paper, an efficient decremented learning algorithm to prune redundant nodes is deduced from the incremental learning algorithm 1 proposed in [9] for added nodes, and two decremented learning algorithms to remove training samples are deduced from the two incremental learning algorithms proposed in [10] for added inputs. The proposed decremented learning algorithm for reduced nodes utilizes the inverse Cholesterol factor of the Herminia matrix in the ridge inverse, to update the output weights recursively, as the incremental learning algorithm 1 for added nodes in [9], while that inverse Cholesterol factor is updated with an unitary transformation. The proposed decremented learning algorithm 1 for reduced inputs updates the output weights recursively with the inverse of the Herminia matrix in the ridge inverse, and updates that inverse recursively, as the incremental learning algorithm 1 for added inputs in [10].