Communication Efficient Distributed Learning over Wireless Channels
This addresses communication efficiency for distributed learning systems in capacity-constrained wireless networks, representing an incremental improvement.
The paper tackles the heavy communication burden in vertical distributed learning over wireless networks by proposing a hierarchical framework with low-dimensional embeddings and opportunistic carrier sensing-based max-pooling, achieving nearly the same model accuracy as using all raw data while reducing communication load to be independent of the number of workers.
Vertical distributed learning exploits the local features collected by multiple learning workers to form a better global model. However, the exchange of data between the workers and the model aggregator for parameter training incurs a heavy communication burden, especially when the learning system is built upon capacity-constrained wireless networks. In this paper, we propose a novel hierarchical distributed learning framework, where each worker separately learns a low-dimensional embedding of their local observed data. Then, they perform communication efficient distributed max-pooling for efficiently transmitting the synthesized input to the aggregator. For data exchange over a shared wireless channel, we propose an opportunistic carrier sensing-based protocol to implement the max-pooling operation for the output data from all the learning workers. Our simulation experiments show that the proposed learning framework is able to achieve almost the same model accuracy as the learning model using the concatenation of all the raw outputs from the learning workers, while requiring a communication load that is independent of the number of workers.