AVATAR -- Machine Learning Pipeline Evaluation Using Surrogate Model
This addresses a bottleneck in automated machine learning (AutoML) by reducing computational overhead, though it is an incremental improvement over existing optimization techniques.
The paper tackles the time-consuming evaluation of machine learning pipelines in automated composition and optimization by proposing AVATAR, a surrogate model that quickly identifies invalid pipelines without execution, resulting in more efficient exploration of complex pipelines compared to traditional methods.
The evaluation of machine learning (ML) pipelines is essential during automatic ML pipeline composition and optimisation. The previous methods such as Bayesian-based and genetic-based optimisation, which are implemented in Auto-Weka, Auto-sklearn and TPOT, evaluate pipelines by executing them. Therefore, the pipeline composition and optimisation of these methods requires a tremendous amount of time that prevents them from exploring complex pipelines to find better predictive models. To further explore this research challenge, we have conducted experiments showing that many of the generated pipelines are invalid, and it is unnecessary to execute them to find out whether they are good pipelines. To address this issue, we propose a novel method to evaluate the validity of ML pipelines using a surrogate model (AVATAR). The AVATAR enables to accelerate automatic ML pipeline composition and optimisation by quickly ignoring invalid pipelines. Our experiments show that the AVATAR is more efficient in evaluating complex pipelines in comparison with the traditional evaluation approaches requiring their execution.