The VOROS: Lifting ROC curves to 3D
This provides a more robust evaluation metric for binary classifiers in scenarios with imbalanced data or uncertain costs, though it is incremental as it builds on existing ROC generalizations.
The paper tackles the problem that the area under the ROC curve (AUC) poorly captures classifier performance under class imbalance or varying misclassification costs by introducing a new 3D ROC surface and the VOROS volume measure, which naturally incorporates these costs and allows modeling with ranges rather than exact values.
While the area under the ROC curve is perhaps the most common measure that is used to rank the relative performance of different binary classifiers, longstanding field folklore has noted that it can be a measure that ill-captures the benefits of different classifiers when either the actual class values or misclassification costs are highly unbalanced between the two classes. We introduce a new ROC surface, and the VOROS, a volume over this ROC surface, as a natural way to capture these costs, by lifting the ROC curve to 3D. Compared to previous attempts to generalize the ROC curve, our formulation also provides a simple and intuitive way to model the scenario when only ranges, rather than exact values, are known for possible class imbalance and misclassification costs.