Fine-Grained Address Segmentation for Attention-Based Variable-Degree Prefetching
This work addresses the challenge of efficient memory prefetching for computer architecture, offering a significant performance boost over existing methods, though it is incremental in advancing ML-based prefetching techniques.
The paper tackled the problem of improving prefetching performance in memory systems by proposing TransFetch, a novel attention-based model that uses fine-grained address segmentation and delta bitmaps, achieving up to 36% higher F1-score in predictions and 38.75% IPC improvement compared to no prefetching.
Machine learning algorithms have shown potential to improve prefetching performance by accurately predicting future memory accesses. Existing approaches are based on the modeling of text prediction, considering prefetching as a classification problem for sequence prediction. However, the vast and sparse memory address space leads to large vocabulary, which makes this modeling impractical. The number and order of outputs for multiple cache line prefetching are also fundamentally different from text prediction. We propose TransFetch, a novel way to model prefetching. To reduce vocabulary size, we use fine-grained address segmentation as input. To predict unordered sets of future addresses, we use delta bitmaps for multiple outputs. We apply an attention-based network to learn the mapping between input and output. Prediction experiments demonstrate that address segmentation achieves 26% - 36% higher F1-score than delta inputs and 15% - 24% higher F1-score than page & offset inputs for SPEC 2006, SPEC 2017, and GAP benchmarks. Simulation results show that TransFetch achieves 38.75% IPC improvement compared with no prefetching, outperforming the best-performing rule-based prefetcher BOP by 10.44%, and ML-based prefetcher Voyager by 6.64%.