Learning Semantics, Not Addresses: Runtime Neural Prefetching for Far Memory
This addresses performance bottlenecks in far-memory systems for data-intensive applications, representing a novel method rather than an incremental improvement.
The paper tackled the problem of memory prefetching in far-memory systems by introducing FarSight, which decouples application semantics from runtime memory layout to predict access patterns, resulting in up to 3.6x higher performance across four data-intensive workloads.
Memory prefetching has long boosted CPU caches and is increasingly vital for far-memory systems, where large portions of memory are offloaded to cheaper, remote tiers. While effective prefetching requires accurate prediction of future accesses, prior ML approaches have been limited to simulation or small-scale hardware. We introduce FarSight, the first Linux-based far-memory system to leverage deep learning by decoupling application semantics from runtime memory layout. This separation enables offline-trained models to predict access patterns over a compact ordinal vocabulary, which are resolved at runtime through lightweight mappings. Across four data-intensive workloads, FarSight delivers up to 3.6x higher performance than the state-of-the-art.