Stacked Confusion Reject Plots (SCORE)
This work addresses the need for non-experts to understand classifier behavior in critical domains like health and driver assistance, but it is incremental as it builds on existing reject curve methods.
The authors tackled the problem of interpreting classifier certainty in critical applications by proposing Stacked Confusion Reject Plots (SCORE) to replace abstract reject curves, resulting in a more intuitive visualization tool demonstrated on artificial Gaussian data with provided Python code.
Machine learning is more and more applied in critical application areas like health and driver assistance. To minimize the risk of wrong decisions, in such applications it is necessary to consider the certainty of a classification to reject uncertain samples. An established tool for this are reject curves that visualize the trade-off between the number of rejected samples and classification performance metrics. We argue that common reject curves are too abstract and hard to interpret by non-experts. We propose Stacked Confusion Reject Plots (SCORE) that offer a more intuitive understanding of the used data and the classifier's behavior. We present example plots on artificial Gaussian data to document the different options of SCORE and provide the code as a Python package.