CPU Simulation with Ranked Set Sampling and Repeated Subsampling
This work addresses efficiency and accuracy in computer architecture simulation for researchers and engineers, presenting incremental improvements over existing sampling techniques.
The paper tackles the problem of slow computer architecture simulation by improving random sampling methods, showing that ranked set sampling reduces confidence interval width by up to 50% and a repeated subsampling scheme lowers maximum error from 35% to 10%, achieving an average error below 2% in SPEC CPU 2017 benchmarks.
Computer system simulation studies routinely rely on executing a limited number of short application regions, since full end-to-end simulation is prohibitively time-consuming. To preserve representativeness, existing methods employ either random sampling or phase-based characterization to identify representative regions. In this work, we revisit random sampling in the context of computer architecture simulation. To assess how the confidence level varies with different micro-architectural configurations, we examine how the sample standard deviation relates to the sample mean. We show that the ranked set sampling (RSS) technique - well established in the statistical literature - maps naturally to architectural simulation and yields significantly tighter confidence intervals than simple random sampling. Across our experiments, RSS reduces the confidence interval width by up to 50%. We further introduce a repeated subsampling scheme that identifies representative simulation regions for future studies. For a fixed sample size, this approach reduces the maximum observed error from 35% to 10%. Evaluating two selection criteria, we find that more informed subsample selection provides additional accuracy gains. Overall, our method achieves an average error below 2% and a maximum error of 3.5% across individual SPEC CPU 2017 Integer applications when simulating 30 regions of 1 million instructions each.