Collaboration Equilibrium in Federated Learning
This addresses the challenge of optimizing collaboration in federated learning networks to enhance performance while maintaining privacy, representing an incremental improvement over standard approaches.
The paper tackles the problem of distributional discrepancies in federated learning by proposing a 'collaboration equilibrium' where clients form smaller coalitions to maximize model improvement, isolating unhelpful collaborators, and demonstrates effectiveness through experiments on synthetic and real-world datasets.
Federated learning (FL) refers to the paradigm of learning models over a collaborative research network involving multiple clients without sacrificing privacy. Recently, there have been rising concerns on the distributional discrepancies across different clients, which could even cause counterproductive consequences when collaborating with others. While it is not necessarily that collaborating with all clients will achieve the best performance, in this paper, we study a rational collaboration called ``collaboration equilibrium'' (CE), where smaller collaboration coalitions are formed. Each client collaborates with certain members who maximally improve the model learning and isolates the others who make little contribution. We propose the concept of benefit graph which describes how each client can benefit from collaborating with other clients and advance a Pareto optimization approach to identify the optimal collaborators. Then we theoretically prove that we can reach a CE from the benefit graph through an iterative graph operation. Our framework provides a new way of setting up collaborations in a research network. Experiments on both synthetic and real world data sets are provided to demonstrate the effectiveness of our method.