FedLog: Personalized Federated Classification with Less Communication and More Flexibility
This addresses communication efficiency for federated learning systems, but it is incremental as it builds on existing methods with privacy extensions.
The paper tackled the problem of high communication overhead in federated representation learning by proposing FedLog, which shares data summaries instead of model parameters and uses Bayesian inference for aggregation, achieving high learning accuracy with reduced communication.
Federated representation learning (FRL) aims to learn personalized federated models with effective feature extraction from local data. FRL algorithms that share the majority of the model parameters face significant challenges with huge communication overhead. This overhead stems from the millions of neural network parameters and slow aggregation progress of the averaging heuristic. To reduce the overhead, we propose to share sufficient data summaries instead of raw model parameters. The data summaries encode minimal sufficient statistics of an exponential family, and Bayesian inference is utilized for global aggregation. It helps to reduce message sizes and communication frequency. To further ensure formal privacy guarantee, we extend it with differential privacy framework. Empirical results demonstrate high learning accuracy with low communication overhead of our method.