Predicting the Performance of a Computing System with Deep Networks
This addresses the challenge of hardware procurement and scaling for modern applications by providing a method to predict performance without direct benchmarking, though it is incremental as it applies existing deep learning techniques to a new domain.
The paper tackles the problem of predicting computing system performance and energy consumption by building deep learning models to estimate benchmark scores for unseen hardware, achieving high R^2 scores of 0.96, 0.98, and 0.94 with three different neural networks.
Predicting the performance and energy consumption of computing hardware is critical for many modern applications. This will inform procurement decisions, deployment decisions, and autonomic scaling. Existing approaches to understanding the performance of hardware largely focus around benchmarking -- leveraging standardised workloads which seek to be representative of an end-user's needs. Two key challenges are present; benchmark workloads may not be representative of an end-user's workload, and benchmark scores are not easily obtained for all hardware. Within this paper, we demonstrate the potential to build Deep Learning models to predict benchmark scores for unseen hardware. We undertake our evaluation with the openly available SPEC 2017 benchmark results. We evaluate three different networks, one fully-connected network along with two Convolutional Neural Networks (one bespoke and one ResNet inspired) and demonstrate impressive $R^2$ scores of 0.96, 0.98 and 0.94 respectively.