Cost-Sensitive Stacking: an Empirical Evaluation
This work addresses cost-sensitive learning for stacking ensembles, which is important for real-world classification problems with varying misclassification costs, but it is incremental as it builds on existing ensemble methods without introducing a new paradigm.
The paper tackles the problem of cost-sensitive stacking in classification, where misclassification costs vary between instances, by empirically evaluating the appropriate setup for cost-sensitive stacking ensembles. The experiments on twelve datasets with real, instance-dependent costs show that best performance is achieved when both levels of stacking use cost-sensitive classification decisions.
Many real-world classification problems are cost-sensitive in nature, such that the misclassification costs vary between data instances. Cost-sensitive learning adapts classification algorithms to account for differences in misclassification costs. Stacking is an ensemble method that uses predictions from several classifiers as the training data for another classifier, which in turn makes the final classification decision. While a large body of empirical work exists where stacking is applied in various domains, very few of these works take the misclassification costs into account. In fact, there is no consensus in the literature as to what cost-sensitive stacking is. In this paper we perform extensive experiments with the aim of establishing what the appropriate setup for a cost-sensitive stacking ensemble is. Our experiments, conducted on twelve datasets from a number of application domains, using real, instance-dependent misclassification costs, show that for best performance, both levels of stacking require cost-sensitive classification decision.