Developing parsimonious ensembles using ensemble diversity within a reinforcement learning framework
This work addresses the challenge of creating parsimonious and interpretable ensembles for predictive modeling, representing an incremental improvement by explicitly integrating diversity into existing ensemble selection methods.
The paper tackled the problem of building accurate and interpretable predictive ensembles by developing algorithms that incorporate ensemble diversity into a reinforcement learning framework for ensemble selection, resulting in ensembles that were more accurate and significantly smaller in size than those ignoring diversity.
Heterogeneous ensembles built from the predictions of a wide variety and large number of diverse base predictors represent a potent approach to building predictive models for problems where the ideal base/individual predictor may not be obvious. Ensemble selection is an especially promising approach here, not only for improving prediction performance, but also because of its ability to select a collectively predictive subset, often a relatively small one, of the base predictors. In this paper, we present a set of algorithms that explicitly incorporate ensemble diversity, a known factor influencing predictive performance of ensembles, into a reinforcement learning framework for ensemble selection. We rigorously tested these approaches on several challenging problems and associated data sets, yielding that several of them produced more accurate ensembles than those that don't explicitly consider diversity. More importantly, these diversity-incorporating ensembles were much smaller in size, i.e., more parsimonious, than the latter types of ensembles. This can eventually aid the interpretation or reverse engineering of predictive models assimilated into the resultant ensemble(s).