TEP-GNN: Accurate Execution Time Prediction of Functional Tests using Graph Neural Networks
This addresses performance prediction for software developers, but it is incremental as it focuses on a specific case and requires generalization to unseen projects.
The paper tackles the problem of predicting unit test execution times by proposing TEP-GNN, a model using flow-augmented ASTs and graph neural networks, achieving a Pearson correlation of 0.789 on real-life Java programs.
Predicting the performance of production code prior to actually executing or benchmarking it is known to be highly challenging. In this paper, we propose a predictive model, dubbed TEP-GNN, which demonstrates that high-accuracy performance prediction is possible for the special case of predicting unit test execution times. TEP-GNN uses FA-ASTs, or flow-augmented ASTs, as a graph-based code representation approach, and predicts test execution times using a powerful graph neural network (GNN) deep learning model. We evaluate TEP-GNN using four real-life Java open source programs, based on 922 test files mined from the projects' public repositories. We find that our approach achieves a high Pearson correlation of 0.789, considerable outperforming a baseline deep learning model. However, we also find that more work is needed for trained models to generalize to unseen projects. Our work demonstrates that FA-ASTs and GNNs are a feasible approach for predicting absolute performance values, and serves as an important intermediary step towards being able to predict the performance of arbitrary code prior to execution.