Efficient Inverse-Free Incremental and Decremental Algorithms for Multiple Hidden Nodes in Extreme Learning Machine
This work addresses efficiency improvements for ELM training in machine learning applications, but it is incremental as it builds on existing inverse-free recursive algorithms.
The paper tackles the computational complexity of updating extreme learning machines (ELMs) by proposing inverse-free algorithms for both adding and removing multiple hidden nodes per iteration, with Tikhonov regularization. The result includes efficient incremental and decremental algorithms that reduce computational overhead compared to prior methods.
The inverse-free extreme learning machine (ELM) algorithm proposed in [4] was based on an inverse-free algorithm to compute the regularized pseudo-inverse, which was deduced from an inverse-free recursive algorithm to update the inverse of a Hermitian matrix. Before that recursive algorithm was applied in [4], its improved version had been utilized in previous literatures [9], [10]. Accordingly from the improved recursive algorithm [9], [10], several efficient inverse-free algorithms for ELM were proposed in [13] to reduce the computational complexity. In this paper, we propose two inverse-free algorithms for ELM with Tikhonov regularization, which can increase multiple hidden nodes in an iteration. On the other hand, we also propose two efficient decremental learning algorithms for ELM with Tikhonov regularization, which can remove multiple redundant nodes in an iteration.