Deterministic Online Classification: Non-iteratively Reweighted Recursive Least-Squares for Binary Class Rebalancing
This work addresses computational efficiency for online classification in machine learning, though it is incremental as it builds on existing weighted least-squares methods.
The paper tackles the problem of high computational cost in online weighted least-squares classification by introducing a deterministic algorithm with constant time complexity for binary class rebalancing, demonstrating exact convergence to batch formulation and outperforming state-of-the-art stochastic methods on real-world datasets.
Deterministic solutions are becoming more critical for interpretability. Weighted Least-Squares (WLS) has been widely used as a deterministic batch solution with a specific weight design. In the online settings of WLS, exact reweighting is necessary to converge to its batch settings. In order to comply with its necessity, the iteratively reweighted least-squares algorithm is mainly utilized with a linearly growing time complexity which is not attractive for online learning. Due to the high and growing computational costs, an efficient online formulation of reweighted least-squares is desired. We introduce a new deterministic online classification algorithm of WLS with a constant time complexity for binary class rebalancing. We demonstrate that our proposed online formulation exactly converges to its batch formulation and outperforms existing state-of-the-art stochastic online binary classification algorithms in real-world data sets empirically.