Federated Online Learning for Heterogeneous Multisource Streaming Data
This work addresses the problem of handling high-dimensional, heterogeneous streaming data in federated settings for applications like finance and web analytics, representing an incremental advance by combining federated and online learning with personalization.
The paper tackles the challenge of analyzing distributed multi-source streaming data under privacy constraints by proposing a federated online learning method that constructs personalized models for each source and uses a subgroup assumption to capture similarities, achieving optimal statistical efficiency and advantageous prediction performance on financial lending and web log data.
Federated learning has emerged as an essential paradigm for distributed multi-source data analysis under privacy concerns. Most existing federated learning methods focus on the ``static" datasets. However, in many real-world applications, data arrive continuously over time, forming streaming datasets. This introduces additional challenges for data storage and algorithm design, particularly under high-dimensional settings. In this paper, we propose a federated online learning (FOL) method for distributed multi-source streaming data analysis. To account for heterogeneity, a personalized model is constructed for each data source, and a novel ``subgroup" assumption is employed to capture potential similarities, thereby enhancing model performance. We adopt the penalized renewable estimation method and the efficient proximal gradient descent for model training. The proposed method aligns with both federated and online learning frameworks: raw data are not exchanged among sources, ensuring data privacy, and only summary statistics of previous data batches are required for model updates, significantly reducing storage demands. Theoretically, we establish the consistency properties for model estimation, variable selection, and subgroup structure recovery, demonstrating optimal statistical efficiency. Simulations illustrate the effectiveness of the proposed method. Furthermore, when applied to the financial lending data and the web log data, the proposed method also exhibits advantageous prediction performance. Results of the analysis also provide some practical insights.