Graph Federated Learning with Hidden Representation Sharing
This addresses privacy concerns in multi-client systems like social networks where data sharing is needed for learning but restricted by FL, offering a solution for incremental improvement in graph-based federated settings.
The paper tackles the conflict between Learning on Graphs (LoG) and Federated Learning (FL) by proposing Graph Federated Learning (GFL), which shares hidden representations instead of raw data to protect privacy, and demonstrates a good match between theory and practice in classification tasks.
Learning on Graphs (LoG) is widely used in multi-client systems when each client has insufficient local data, and multiple clients have to share their raw data to learn a model of good quality. One scenario is to recommend items to clients with limited historical data and sharing similar preferences with other clients in a social network. On the other hand, due to the increasing demands for the protection of clients' data privacy, Federated Learning (FL) has been widely adopted: FL requires models to be trained in a multi-client system and restricts sharing of raw data among clients. The underlying potential data-sharing conflict between LoG and FL is under-explored and how to benefit from both sides is a promising problem. In this work, we first formulate the Graph Federated Learning (GFL) problem that unifies LoG and FL in multi-client systems and then propose sharing hidden representation instead of the raw data of neighbors to protect data privacy as a solution. To overcome the biased gradient problem in GFL, we provide a gradient estimation method and its convergence analysis under the non-convex objective. In experiments, we evaluate our method in classification tasks on graphs. Our experiment shows a good match between our theory and the practice.