Green Machine Learning via Augmented Gaussian Processes and Multi-Information Source Optimization
This work addresses the energy and computational cost challenges in machine learning for researchers and practitioners, though it is incremental as it builds on existing Bayesian optimization and multi-source strategies.
The paper tackles the problem of reducing computational time and energy in machine learning model search by proposing an Augmented Gaussian Process method with multi-information source optimization (AGP-MISO), which uses a smaller dataset portion to optimize SVM hyperparameters, achieving results comparable to traditional methods but with reduced resource usage.
Searching for accurate Machine and Deep Learning models is a computationally expensive and awfully energivorous process. A strategy which has been gaining recently importance to drastically reduce computational time and energy consumed is to exploit the availability of different information sources, with different computational costs and different "fidelity", typically smaller portions of a large dataset. The multi-source optimization strategy fits into the scheme of Gaussian Process based Bayesian Optimization. An Augmented Gaussian Process method exploiting multiple information sources (namely, AGP-MISO) is proposed. The Augmented Gaussian Process is trained using only "reliable" information among available sources. A novel acquisition function is defined according to the Augmented Gaussian Process. Computational results are reported related to the optimization of the hyperparameters of a Support Vector Machine (SVM) classifier using two sources: a large dataset - the most expensive one - and a smaller portion of it. A comparison with a traditional Bayesian Optimization approach to optimize the hyperparameters of the SVM classifier on the large dataset only is reported.