A Skew-Sensitive Evaluation Framework for Imbalanced Data Classification
This work addresses the challenge of fair assessment in imbalanced data classification, which is crucial for domains like healthcare or finance, but it appears incremental as it builds on existing metrics like Balanced Accuracy.
The paper tackles the problem of evaluating classifiers on imbalanced datasets where existing metrics fail to account for varying class importance, proposing a new framework that is sensitive to skews in class cardinalities and importances. Experiments on real-world datasets across three domains demonstrate its effectiveness in both evaluating and training classifiers.
Class distribution skews in imbalanced datasets may lead to models with prediction bias towards majority classes, making fair assessment of classifiers a challenging task. Metrics such as Balanced Accuracy are commonly used to evaluate a classifier's prediction performance under such scenarios. However, these metrics fall short when classes vary in importance. In this paper, we propose a simple and general-purpose evaluation framework for imbalanced data classification that is sensitive to arbitrary skews in class cardinalities and importances. Experiments with several state-of-the-art classifiers tested on real-world datasets from three different domains show the effectiveness of our framework - not only in evaluating and ranking classifiers, but also training them.