Broad stochastic configuration residual learning system for norm-convergent universal approximation
This work addresses a foundational problem in neural network theory by improving the robustness of universal approximation for randomized learning models, though it is incremental as it builds on the Broad Residual Learning System framework.
The paper tackled the issue of norm convergence in randomized learning networks by proposing the Broad Stochastic Configuration Residual Learning System (BSCRLS), which introduces a supervisory mechanism to constrain random parameters, proving its universal approximation theorem with norm convergence and demonstrating effectiveness in solar panel dust detection experiments compared to 13 other algorithms.
Universal approximation serves as the foundation of neural network learning algorithms. However, some networks establish their universal approximation property by demonstrating that the iterative errors converge in probability measure rather than the more rigorous norm convergence, which makes the universal approximation property of randomized learning networks highly sensitive to random parameter selection, Broad residual learning system (BRLS), as a member of randomized learning models, also encounters this issue. We theoretically demonstrate the limitation of its universal approximation property, that is, the iterative errors do not satisfy norm convergence if the selection of random parameters is inappropriate and the convergence rate meets certain conditions. To address this issue, we propose the broad stochastic configuration residual learning system (BSCRLS) algorithm, which features a novel supervisory mechanism adaptively constraining the range settings of random parameters on the basis of BRLS framework, Furthermore, we prove the universal approximation theorem of BSCRLS based on the more stringent norm convergence. Three versions of incremental BSCRLS algorithms are presented to satisfy the application requirements of various network updates. Solar panels dust detection experiments are performed on publicly available dataset and compared with 13 deep and broad learning algorithms. Experimental results reveal the effectiveness and superiority of BSCRLS algorithms.