A Hitchhiker's Guide to Understanding Performances of Two-Class Classifiers
This provides a more nuanced tool for researchers and practitioners to compare classifiers based on diverse requirements, though it is incremental as it builds on existing visualization concepts.
The paper tackles the problem of comparing two-class classifiers by introducing the Tile visualization tool, which organizes infinite ranking scores into a 2D map to capture application-specific nuances, and demonstrates its effectiveness by analyzing 74 state-of-the-art semantic segmentation models across four user scenarios.
Properly understanding the performances of classifiers is essential in various scenarios. However, the literature often relies only on one or two standard scores to compare classifiers, which fails to capture the nuances of application-specific requirements. The Tile is a recently introduced visualization tool organizing an infinity of ranking scores into a 2D map. Thanks to the Tile, it is now possible to compare classifiers efficiently, displaying all possible application-specific preferences instead of having to rely on a pair of scores. This hitchhiker's guide to understanding the performances of two-class classifiers presents four scenarios showcasing different user profiles: a theoretical analyst, a method designer, a benchmarker, and an application developer. We introduce several interpretative flavors adapted to the user's needs by mapping different values on the Tile. We illustrate this guide by ranking and analyzing the performances of 74 state-of-the-art semantic segmentation models through the perspective of the four scenarios. Through these user profiles, we demonstrate that the Tile effectively captures the behavior of classifiers in a single visualization, while accommodating an infinite number of ranking scores. Code for mapping the different Tile flavors is available in supplementary material.