Trends and Advancements in Deep Neural Network Communication
It reviews methods for improving communication efficiency and privacy in distributed machine learning, which is incremental as it summarizes existing trends.
The paper provides an overview of recent advancements in distributed neural network communication, addressing challenges like limited bandwidth and privacy in mobile and IoT settings, but does not present specific results or numbers.
Due to their great performance and scalability properties neural networks have become ubiquitous building blocks of many applications. With the rise of mobile and IoT, these models now are also being increasingly applied in distributed settings, where the owners of the data are separated by limited communication channels and privacy constraints. To address the challenges of these distributed environments, a wide range of training and evaluation schemes have been developed, which require the communication of neural network parametrizations. These novel approaches, which bring the "intelligence to the data" have many advantages over traditional cloud solutions such as privacy-preservation, increased security and device autonomy, communication efficiency and high training speed. This paper gives an overview over the recent advancements and challenges in this new field of research at the intersection of machine learning and communications.