TransforMAP: Transformer for Memory Access Prediction
This work addresses memory latency issues in computing systems, offering a novel method that improves prefetching accuracy and performance, though it is incremental in applying Transformers to a specific hardware optimization task.
The paper tackles the problem of memory access prediction for data prefetching by introducing TransforMAP, a Transformer-based model that learns from the entire address space and predicts multiple cache lines, achieving a 35.67% MPKI improvement and 20.55% IPC improvement over state-of-the-art prefetchers.
Data Prefetching is a technique that can hide memory latency by fetching data before it is needed by a program. Prefetching relies on accurate memory access prediction, to which task machine learning based methods are increasingly applied. Unlike previous approaches that learn from deltas or offsets and perform one access prediction, we develop TransforMAP, based on the powerful Transformer model, that can learn from the whole address space and perform multiple cache line predictions. We propose to use the binary of memory addresses as model input, which avoids information loss and saves a token table in hardware. We design a block index bitmap to collect unordered future page offsets under the current page address as learning labels. As a result, our model can learn temporal patterns as well as spatial patterns within a page. In a practical implementation, this approach has the potential to hide prediction latency because it prefetches multiple cache lines likely to be used in a long horizon. We show that our approach achieves 35.67% MPKI improvement and 20.55% IPC improvement in simulation, higher than state-of-the-art Best-Offset prefetcher and ISB prefetcher.