Towards a Prediction of Machine Learning Training Time to Support Continuous Learning Systems Development
This addresses the need for efficient model selection in continuous learning systems, but it is incremental as it critiques an existing method without proposing a new solution.
The paper tackles the problem of predicting machine learning model training time to support automated model selection in MLOps, finding through an empirical study that the Full Parameter Time Complexity approach is not generalizable and heavily context-dependent.
The problem of predicting the training time of machine learning (ML) models has become extremely relevant in the scientific community. Being able to predict a priori the training time of an ML model would enable the automatic selection of the best model both in terms of energy efficiency and in terms of performance in the context of, for instance, MLOps architectures. In this paper, we present the work we are conducting towards this direction. In particular, we present an extensive empirical study of the Full Parameter Time Complexity (FPTC) approach by Zheng et al., which is, to the best of our knowledge, the only approach formalizing the training time of ML models as a function of both dataset's and model's parameters. We study the formulations proposed for the Logistic Regression and Random Forest classifiers, and we highlight the main strengths and weaknesses of the approach. Finally, we observe how, from the conducted study, the prediction of training time is strictly related to the context (i.e., the involved dataset) and how the FPTC approach is not generalizable.