Algorithms for the Communication of Samples
This work addresses communication efficiency in machine learning applications like neural compression and differential privacy, but appears incremental as it builds on existing approaches.
The paper tackles the problem of efficiently communicating noisy data, known as reverse channel coding or channel simulation, by proposing two new coding schemes: ordered random coding (ORC) reduces coding costs, and a hybrid scheme uses dithered quantization for bounded support distributions.
The efficient communication of noisy data has applications in several areas of machine learning, such as neural compression or differential privacy, and is also known as reverse channel coding or the channel simulation problem. Here we propose two new coding schemes with practical advantages over existing approaches. First, we introduce ordered random coding (ORC) which uses a simple trick to reduce the coding cost of previous approaches. This scheme further illuminates a connection between schemes based on importance sampling and the so-called Poisson functional representation. Second, we describe a hybrid coding scheme which uses dithered quantization to more efficiently communicate samples from distributions with bounded support.