Towards Continually Learning Application Performance Models
This addresses performance model reliability for HPC job scheduling and optimization, but it is incremental as it builds on existing continual learning approaches.
The paper tackled the problem of data distribution drift in machine learning-based performance models for HPC systems due to hardware and software changes, resulting in a model that retains accuracy and achieves a 2x improvement in prediction accuracy compared to naive methods.
Machine learning-based performance models are increasingly being used to build critical job scheduling and application optimization decisions. Traditionally, these models assume that data distribution does not change as more samples are collected over time. However, owing to the complexity and heterogeneity of production HPC systems, they are susceptible to hardware degradation, replacement, and/or software patches, which can lead to drift in the data distribution that can adversely affect the performance models. To this end, we develop continually learning performance models that account for the distribution drift, alleviate catastrophic forgetting, and improve generalizability. Our best model was able to retain accuracy, regardless of having to learn the new distribution of data inflicted by system changes, while demonstrating a 2x improvement in the prediction accuracy of the whole data sequence in comparison to the naive approach.