Who to Trust? Aggregating Client Knowledge in Logit-Based Federated Learning
This work addresses communication efficiency and robustness in federated learning for distributed systems, but it is incremental as it builds on existing logit-based methods.
The paper tackled the challenge of aggregating client knowledge in logit-based federated learning to reduce communication costs and handle heterogeneous data, achieving accuracy competitive with centralized training on MNIST and CIFAR-10.
Federated learning (FL) usually shares model weights or gradients, which is costly for large models. Logit-based FL reduces this cost by sharing only logits computed on a public proxy dataset. However, aggregating information from heterogeneous clients is still challenging. This paper studies this problem, introduces and compares three logit aggregation methods: simple averaging, uncertainty-weighted averaging, and a learned meta-aggregator. Evaluated on MNIST and CIFAR-10, these methods reduce communication overhead, improve robustness under non-IID data, and achieve accuracy competitive with centralized training.