Structured Codes for Distributed Matrix Multiplication
It provides fundamental performance limits for distributed computing of bilinear functions, a crucial class for applications like distributed matrix multiplication, with tight bounds for large field sizes.
This work establishes bounds on the optimal sum rate for distributed computing of bilinear functions (e.g., matrix products) from correlated sources, achieving tight bounds for large field sizes and demonstrating unbounded compression gains over Slepian-Wolf coding.
Our work addresses the well-known open problem of distributed computing of bilinear functions of two correlated sources ${\bf A}$ and ${\bf B}$. In a setting with two nodes, with the first node having access to ${\bf A}$ and the second to ${\bf B}$, we establish bounds on the optimal sum rate that allows a receiver to compute an important class of non-linear functions, and in particular bilinear functions, including dot products $\langle {\bf A},{\bf B}\rangle$, and general matrix products ${\bf A}^{\intercal}{\bf B}$ over finite fields. The bounds are tight for large field sizes, for which case we can derive the exact fundamental performance limits for all problem dimensions and a large class of sources. Our achievability scheme involves the design of non-linear transformations of ${\bf A}$ and ${\bf B}$, carefully calibrated to work synergistically with the structured linear encoding scheme by Körner and Marton. The subsequent converses derived here, calibrate the Han-Kobayashi approach and the strong converse of Ahlswede-Gács-Körner to yield relatively tight converses on the sum rate. We exhibit unbounded compression gains over Slepian-Wolf coding, depending on the source correlations. In the end, this work characterizes the fundamental limits of distributed computing for a crucial class of functions, while succinctly capturing the inherent computation structures and source correlations.