Emerging Trends in Federated Learning: From Model Fusion to Federated X Learning
This is an incremental survey paper for researchers in federated learning, summarizing existing trends without presenting new results.
This paper surveys federated learning integration with other algorithms, reviewing improvements to federated averaging and model fusion methods, and discussing federated X learning across multitask, meta-learning, and other paradigms.
Federated learning is a new learning paradigm that decouples data collection and model training via multi-party computation and model aggregation. As a flexible learning setting, federated learning has the potential to integrate with other learning frameworks. We conduct a focused survey of federated learning in conjunction with other learning algorithms. Specifically, we explore various learning algorithms to improve the vanilla federated averaging algorithm and review model fusion methods such as adaptive aggregation, regularization, clustered methods, and Bayesian methods. Following the emerging trends, we also discuss federated learning in the intersection with other learning paradigms, termed federated X learning, where X includes multitask learning, meta-learning, transfer learning, unsupervised learning, and reinforcement learning. In addition to reviewing state-of-the-art studies, this paper also identifies key challenges and applications in this field, while also highlighting promising future directions.