A Communication-Efficient Adaptive Algorithm for Federated Learning under Cumulative Regret
This work addresses communication efficiency in federated learning for distributed systems, though it appears incremental as it builds on existing regret and cost frameworks.
The paper tackled the problem of online stochastic optimization in federated learning by developing a distributed algorithm that achieves order-optimal cumulative regret with low communication cost in bits, contrasting with existing methods focused on offline simple regret and separate communication measures.
We consider the problem of online stochastic optimization in a distributed setting with $M$ clients connected through a central server. We develop a distributed online learning algorithm that achieves order-optimal cumulative regret with low communication cost measured in the total number of bits transmitted over the entire learning horizon. This is in contrast to existing studies which focus on the offline measure of simple regret for learning efficiency. The holistic measure for communication cost also departs from the prevailing approach that \emph{separately} tackles the communication frequency and the number of bits in each communication round.