OptABC: an Optimal Hyperparameter Tuning Approach for Machine Learning Algorithms
This work addresses hyperparameter tuning efficiency for machine learning practitioners, but it is incremental as it builds on existing swarm intelligence methods.
The authors tackled the problem of slow convergence in hyperparameter tuning for machine learning by proposing OptABC, a novel algorithm that integrates multiple techniques to enhance the Artificial Bee Colony method, resulting in faster convergence without significantly decreasing accuracy, as validated by comparisons with state-of-the-art approaches.
Hyperparameter tuning in machine learning algorithms is a computationally challenging task due to the large-scale nature of the problem. In order to develop an efficient strategy for hyper-parameter tuning, one promising solution is to use swarm intelligence algorithms. Artificial Bee Colony (ABC) optimization lends itself as a promising and efficient optimization algorithm for this purpose. However, in some cases, ABC can suffer from a slow convergence rate or execution time due to the poor initial population of solutions and expensive objective functions. To address these concerns, a novel algorithm, OptABC, is proposed to help ABC algorithm in faster convergence toward a near-optimum solution. OptABC integrates artificial bee colony algorithm, K-Means clustering, greedy algorithm, and opposition-based learning strategy for tuning the hyper-parameters of different machine learning models. OptABC employs these techniques in an attempt to diversify the initial population, and hence enhance the convergence ability without significantly decreasing the accuracy. In order to validate the performance of the proposed method, we compare the results with previous state-of-the-art approaches. Experimental results demonstrate the effectiveness of the OptABC compared to existing approaches in the literature.