Whit Schonbein, Matthew G. F. Dosanjh, Scott Levy
In this paper, we present Remote Access Memory Channels (RAMC), an explicit one-sided communication library designed to leverage the capabilities of HPE Cray Slingshot network hardware. Existing one-sided communication frameworks, such as MPI RMA and OpenSHMEM, rely on monolithic shared memory models that introduce scalability and usability challenges. These frameworks often assume symmetric memory regions or require blocking collective operations for window creation, which can mismatch user communication needs and hinder performance. Implicit models, such as PGAS and UPC, aim to simplify programming by treating local and remote memory as a unified region but ultimately rely on explicit mechanisms to implement data movement. MPI's recently-introduced partitioned communication API offers a persistent point-to-point interface but sacrifices the dynamic flexibility of RDMA. RAMC is designed to address these limitations. Based on the core concept of a persistent uni-directional communication channel, RAMC leverages Slingshot's unique memory region counters to enable efficient completion notification. Experiments with a RAMC-based heat diffusion code demonstrate RAMC has no difficulty scaling to 19.6 thousand processes across 250 nodes, and microbenchmark studies across multiple libfabric versions show RAMC can outperform Cray's proprietary MPI implementation (e.g., increases in bandwidth ranging from approx. 100%-130% for 1B-4KiB messages under libfabric 1.15.2, and from approx. 30%-45% under libfabric 2.3.1) while identifying additional areas for improvement, such as small message latencies.