Personalized Federated Learning with Local Attention
This addresses the feature shift problem in federated learning for clients with heterogeneous data, offering a flexible solution that can be integrated into existing methods without extra communication costs, though it is incremental as it builds on prior work on personalization.
The paper tackles the challenge of feature shift in federated learning by proposing pFedLA, a method that incorporates client-specific attention modules into personalized models, which boosts the performance of state-of-the-art FL methods on tasks like image classification and object detection, achieving improvements of up to 5% in accuracy.
Federated Learning (FL) aims to learn a single global model that enables the central server to help the model training in local clients without accessing their local data. The key challenge of FL is the heterogeneity of local data in different clients, such as heterogeneous label distribution and feature shift, which could lead to significant performance degradation of the learned models. Although many studies have been proposed to address the heterogeneous label distribution problem, few studies attempt to explore the feature shift issue. To address this issue, we propose a simple yet effective algorithm, namely \textbf{p}ersonalized \textbf{Fed}erated learning with \textbf{L}ocal \textbf{A}ttention (pFedLA), by incorporating the attention mechanism into personalized models of clients while keeping the attention blocks client-specific. Specifically, two modules are proposed in pFedLA, i.e., the personalized single attention module and the personalized hybrid attention module. In addition, the proposed pFedLA method is quite flexible and general as it can be incorporated into any FL method to improve their performance without introducing additional communication costs. Extensive experiments demonstrate that the proposed pFedLA method can boost the performance of state-of-the-art FL methods on different tasks such as image classification and object detection tasks.