Online-BLS: An Accurate and Efficient Online Broad Learning System for Data Stream Classification
This work addresses the need for accurate and efficient online learning models for data stream classification, representing an incremental improvement over existing methods.
The paper tackles the problem of suboptimal model weights in online learning by introducing an online broad learning system with closed-form solutions, achieving superior accuracy and efficiency on real-world datasets and outperforming state-of-the-art baselines in data stream scenarios with concept drift.
The state-of-the-art online learning models generally conduct a single online gradient descent when a new sample arrives and thus suffer from suboptimal model weights. To this end, we introduce an online broad learning system framework with closed-form solutions for each online update. Different from employing existing incremental broad learning algorithms for online learning tasks, which tend to incur degraded accuracy and expensive online update overhead, we design an effective weight estimation algorithm and an efficient online updating strategy to remedy the above two deficiencies, respectively. Specifically, an effective weight estimation algorithm is first developed by replacing notorious matrix inverse operations with Cholesky decomposition and forward-backward substitution to improve model accuracy. Second, we devise an efficient online updating strategy that dramatically reduces online update time. Theoretical analysis exhibits the splendid error bound and low time complexity of our model. The most popular test-then-training evaluation experiments on various real-world datasets prove its superiority and efficiency. Furthermore, our framework is naturally extended to data stream scenarios with concept drift and exceeds state-of-the-art baselines.