Statistical Estimation and Inference via Local SGD in Federated Learning
This work addresses the problem of performing reliable statistical inference in federated learning for edge computing applications, representing an incremental advancement by adapting existing methods to this setting.
The paper tackles statistical estimation and inference in federated learning with heterogeneous data by analyzing Local SGD, establishing a functional central limit theorem for its iterates and proposing two communication-efficient inference methods (plug-in and random scaling). The results demonstrate that Local SGD achieves both statistical and communication efficiency, with empirical validation.
Federated Learning (FL) makes a large amount of edge computing devices (e.g., mobile phones) jointly learn a global model without data sharing. In FL, data are generated in a decentralized manner with high heterogeneity. This paper studies how to perform statistical estimation and inference in the federated setting. We analyze the so-called Local SGD, a multi-round estimation procedure that uses intermittent communication to improve communication efficiency. We first establish a {\it functional central limit theorem} that shows the averaged iterates of Local SGD weakly converge to a rescaled Brownian motion. We next provide two iterative inference methods: the {\it plug-in} and the {\it random scaling}. Random scaling constructs an asymptotically pivotal statistic for inference by using the information along the whole Local SGD path. Both the methods are communication efficient and applicable to online data. Our theoretical and empirical results show that Local SGD simultaneously achieves both statistical efficiency and communication efficiency.