Stop talking to me -- a communication-avoiding ADER-DG realisation
This work addresses memory and communication bottlenecks in high-order finite volume/difference schemes for computational scientists, offering a principled approach to reduce data movement.
The paper presents a communication-avoiding formulation of ADER-DG for hyperbolic PDEs that reduces memory footprint, achieves single-touch semantics (each degree of freedom read once per time step), and hides distributed memory transfers behind computation, improving performance on modern architectures.
We present a communication- and data-sensitive formulation of ADER-DG for hyperbolic differential equation systems. Sensitive here has multiple flavours: First, the formulation reduces the persistent memory footprint. This reduces pressure on the memory subsystem. Second, the formulation realises the underlying predictor-corrector scheme with single-touch semantics, i.e., each degree of freedom is read on average only once per time step from the main memory. This reduces communication through the memory controllers. Third, the formulation breaks up the tight coupling of the explicit time stepping's algorithmic steps to mesh traversals. This averages out data access peaks. Different operations and algorithmic steps are ran on different grid entities. Finally, the formulation hides distributed memory data transfer behind the computation aligned with the mesh traversal. This reduces pressure on the machine interconnects. All techniques applied by our formulation are elaborated by means of a rigorous task formalism. They break up ADER-DG's tight causal coupling of compute steps and can be generalised to other predictor-corrector schemes.