Fisher-Informed Parameterwise Aggregation for Federated Learning with Heterogeneous Data
This addresses client drift in federated learning for applications with heterogeneous data distributions, representing an incremental improvement over existing aggregation methods.
The paper tackled the problem of client drift in federated learning with non-IID data by proposing Fisher-Informed Parameterwise Aggregation (FIPA), which uses parameter-specific Fisher Information Matrix weights instead of uniform scalar weights, resulting in consistent improvements over averaging-based methods across tasks like regression and image classification.
Federated learning aggregates model updates from distributed clients, but standard first order methods such as FedAvg apply the same scalar weight to all parameters from each client. Under non-IID data, these uniformly weighted updates can be strongly misaligned across clients, causing client drift and degrading the global model. Here we propose Fisher-Informed Parameterwise Aggregation (FIPA), a second-order aggregation method that replaces client-level scalar weights with parameter-specific Fisher Information Matrix (FIM) weights, enabling true parameter-level scaling that captures how each client's data uniquely influences different parameters. With low-rank approximation, FIPA remains communication- and computation-efficient. Across nonlinear function regression, PDE learning, and image classification, FIPA consistently improves over averaging-based aggregation, and can be effectively combined with state-of-the-art client-side optimization algorithms to further improve image classification accuracy. These results highlight the benefits of FIPA for federated learning under heterogeneous data distributions.