StarDist: A Code Generator for Distributed Graph Algorithms
For developers of distributed graph algorithms, StarDist provides a code generation approach that significantly improves performance over existing frameworks.
StarDist is a DSL and analysis-transformation framework that generates high-performant distributed graph algorithms, outperforming DRONE and D-Galois by up to 19x and 7x on high communication setups, and 1.4x and 1.92x on high congestion networks across three algorithms.
We introduce StarDist, a Domain Specific Language for generating high-performant distributed graph algorithms in the message passing model. Our analysis-transformation framework optimizes graph traversal based on graph property access patterns, reduces global lock acquisitions on distributed structures, and minimizes message queues used in reduction operations. We provide a network optimized communication runtime for reduction operations that couples with our analysis framework and optimizes the propagation of updates based on vertex residency. StarDist is able to identify monotonic reduction blocks and is able to fuse reduction iterations over graphs into \textit{pulses}. We evaluate StarDist using three fundamental graph algorithms belonging to the CONGEST model: single-source shortest paths, weakly connected components, and PageRank computation, using a suite comprising both real-world and synthetic graphs across varying densities of topological compaction. Our results illustrate that the code generated with StarDist outperforms the distributed frameworks DRONE and D-Galois by an average of 19$\times$ and 7$\times$, respectively on our high communication setup and by 1.4$\times$ and 1.92$\times$ respectively on our high congestion network setup when averaged across all three algorithms.