Global Convergence of Online Identification for Mixed Linear Regression
It addresses the fundamental challenge of online learning for mixed linear regression, which is incremental as it extends existing offline methods to online settings with global convergence guarantees.
This paper tackles the problem of online identification and data clustering for mixed linear regression models, introducing two new online algorithms based on the expectation-maximization principle that achieve global convergence without relying on traditional i.i.d. data assumptions, with results showing asymptotic equivalence to known parameters in terms of within-cluster error and correct categorization probability.
Mixed linear regression (MLR) is a powerful model for characterizing nonlinear relationships by utilizing a mixture of linear regression sub-models. The identification of MLR is a fundamental problem, where most of the existing results focus on offline algorithms, rely on independent and identically distributed (i.i.d) data assumptions, and provide local convergence results only. This paper investigates the online identification and data clustering problems for two basic classes of MLRs, by introducing two corresponding new online identification algorithms based on the expectation-maximization (EM) principle. It is shown that both algorithms will converge globally without resorting to the traditional i.i.d data assumptions. The main challenge in our investigation lies in the fact that the gradient of the maximum likelihood function does not have a unique zero, and a key step in our analysis is to establish the stability of the corresponding differential equation in order to apply the celebrated Ljung's ODE method. It is also shown that the within-cluster error and the probability that the new data is categorized into the correct cluster are asymptotically the same as those in the case of known parameters. Finally, numerical simulations are provided to verify the effectiveness of our online algorithms.