An efficient multi-GPU implementation for the Discontinuous Galerkin ocean model SLIM
This work addresses the high computational cost of unstructured-mesh ocean models for coastal applications, enabling ultra-high-resolution simulations that were previously infeasible.
The authors present a GPU-optimized implementation of a 3D Discontinuous Galerkin ocean model, achieving performance equivalent to ~1500 CPU cores on a single A100 GPU and ~50x speedup over a 128-core CPU node with 4 GPUs, with weak scaling up to 1024 GPUs. Applied to the Great Barrier Reef, it enables 5x finer resolution than existing models while maintaining a 100x physical-to-numerical time ratio.
Unstructured-mesh ocean models are increasingly used for coastal applications due to their ability to represent complex geometries and apply local grid refinement where needed. However, their broader use has been hindered by their high computational cost, particularly for models based on the Discontinuous Galerkin finite element (DG-FE) method, which involves significantly more degrees of freedom than traditional finite volume or continuous finite element approaches. The rapid emergence of GPU-based high-performance computing architectures now offers a pathway to address this limitation, as DG-FE formulations are inherently well suited to massively parallel, element-wise computations. Here, we present a full 3D DG-FE ocean model implementation optimized for both single- and multi-GPU systems, with support for both NVIDIA and AMD architectures. We detail the computational strategies employed to achieve high performance, including memory layout optimization, kernel-level parallelization, and matrix-free solvers for key vertical processes. Benchmark results demonstrate that a single HPC-grade GPU (e.g. NVIDIA A100) delivers performance equivalent to approximately 1500 CPU cores, while replacing a 128-core CPU node with a 4xA100 GPU node yields a speedup of around 50x. Weak-scaling efficiency is maintained up to 1024 GPUs. We further demonstrate the model's capabilities on a real-world application in the Great Barrier Reef, achieving a spatial resolution five times finer than the most accurate existing model while maintaining a physical-to-numerical time ratio of 100. These results highlight how GPU-accelerated DG-FE methods can dramatically advance the capabilities of unstructured-mesh ocean modeling, enabling ultra-high-resolution coastal simulations that were previously infeasible.