Model selection with Gini indices under auto-calibration
This addresses a methodological issue for researchers and practitioners in statistical modeling, but it is incremental as it builds on known limitations of the Gini index.
The paper tackled the problem of model selection using the Gini index, which can lead to wrong decisions due to its lack of strict consistency, and showed that restricting to auto-calibrated regression models allows for strictly consistent scoring.
The Gini index does not give a strictly consistent scoring rule in general. Therefore, maximizing the Gini index may lead to wrong decisions. The main issue is that the Gini index is a rank-based score that is not calibration-sensitive. We show that the Gini index allows for strictly consistent scoring if we restrict to the class of auto-calibrated regression models.