The Anatomy of Large-Scale Distributed Graph Algorithms
This work addresses the challenge of experimental analysis in distributed graph algorithms for researchers, but it is incremental as it offers recommendations based on existing research.
The paper analyzes how distributed graph algorithms (DGAs) are experimentally evaluated, highlighting that runtime properties are integral to interpreting results, and provides recommendations for better describing DGA outcomes.
The increasing complexity of the software/hardware stack of modern supercomputers results in explosion of parameters. The performance analysis becomes a truly experimental science, even more challenging in the presence of massive irregularity and data dependency. We analyze how the existing body of research handles the experimental aspect in the context of distributed graph algorithms (DGAs). We distinguish algorithm-level contributions, often prioritized by authors, from runtime-level concerns that are harder to place. We show that the runtime is such an integral part of DGAs that experimental results are difficult to interpret and extrapolate without understanding the properties of the runtime used. We argue that in order to gain understanding about the impact of runtimes, more information needs to be gathered. To begin this process, we provide an initial set of recommendations for describing DGA results based on our analysis of the current state of the field.