Personalized Federated Learning with Bidirectional Communication Compression via One-Bit Random Sketching
This addresses communication efficiency and personalization for decentralized data in federated learning, but it is incremental as it builds on existing methods with a novel compression technique.
The paper tackles the challenges of bidirectional communication overhead and client-side data heterogeneity in federated learning by proposing pFed1BS, a personalized framework using one-bit random sketching, which reduces communication costs while achieving competitive performance compared to advanced algorithms.
Federated Learning (FL) enables collaborative training across decentralized data, but faces key challenges of bidirectional communication overhead and client-side data heterogeneity. To address communication costs while embracing data heterogeneity, we propose pFed1BS, a novel personalized federated learning framework that achieves extreme communication compression through one-bit random sketching. In personalized FL, the goal shifts from training a single global model to creating tailored models for each client. In our framework, clients transmit highly compressed one-bit sketches, and the server aggregates and broadcasts a global one-bit consensus. To enable effective personalization, we introduce a sign-based regularizer that guides local models to align with the global consensus while preserving local data characteristics. To mitigate the computational burden of random sketching, we employ the Fast Hadamard Transform for efficient projection. Theoretical analysis guarantees that our algorithm converges to a stationary neighborhood of the global potential function. Numerical simulations demonstrate that pFed1BS substantially reduces communication costs while achieving competitive performance compared to advanced communication-efficient FL algorithms.