MetaStackVis: Visually-Assisted Performance Evaluation of Metamodels
This work addresses the trial-and-error process in selecting metamodels for stacking ensembles, primarily benefiting researchers and practitioners in machine learning, but it is incremental as it builds upon a previous tool (StackGenVis) by extending it to multiple metamodels.
The paper tackles the problem of evaluating the impact of different metamodels on stacking ensemble performance by introducing MetaStackVis, a visualization tool that helps users explore and compare metamodels based on predictive probabilities and validation metrics, with evaluation conducted through a medical dataset usage scenario and expert interviews.
Stacking (or stacked generalization) is an ensemble learning method with one main distinctiveness from the rest: even though several base models are trained on the original data set, their predictions are further used as input data for one or more metamodels arranged in at least one extra layer. Composing a stack of models can produce high-performance outcomes, but it usually involves a trial-and-error process. Therefore, our previously developed visual analytics system, StackGenVis, was mainly designed to assist users in choosing a set of top-performing and diverse models by measuring their predictive performance. However, it only employs a single logistic regression metamodel. In this paper, we investigate the impact of alternative metamodels on the performance of stacking ensembles using a novel visualization tool, called MetaStackVis. Our interactive tool helps users to visually explore different singular and pairs of metamodels according to their predictive probabilities and multiple validation metrics, as well as their ability to predict specific problematic data instances. MetaStackVis was evaluated with a usage scenario based on a medical data set and via expert interviews.