Prediction Instability in Machine Learning Ensembles
This addresses the problem of ensuring safe and explainable use of ensembles for practitioners, highlighting inherent limitations rather than incremental improvements.
The paper proves a theorem showing that any machine learning ensemble must exhibit at least one form of prediction instability, such as ignoring model agreement or being manipulable, and identifies specific instabilities in popular algorithms like random forest and xgboost.
In machine learning ensembles predictions from multiple models are aggregated. Despite widespread use and strong performance of ensembles in applied problems little is known about the mathematical properties of aggregating models and associated consequences for safe, explainable use of such models. In this paper we prove a theorem that shows that any ensemble will exhibit at least one of the following forms of prediction instability. It will either ignore agreement among all underlying models, change its mind when none of the underlying models have done so, or be manipulable through inclusion or exclusion of options it would never actually predict. As a consequence, ensemble aggregation procedures will always need to balance the benefits of information use against the risk of these prediction instabilities. This analysis also sheds light on what specific forms of prediction instability to expect from particular ensemble algorithms; for example popular tree ensembles like random forest, or xgboost will violate basic, intuitive fairness properties. Finally, we show that this can be ameliorated by using consistent models in asymptotic conditions.