Applying the Decisiveness and Robustness Metrics to Convolutional Neural Networks
This work provides incremental clarifications to existing metrics for evaluating neural networks in image classification tasks.
The authors reviewed three classifier quality metrics (geometric accuracy, decisiveness, robustness) and applied them to convolutional neural networks on large datasets like ImageNet, showing examples with AlexNet and DenseNet.
We review three recently-proposed classifier quality metrics and consider their suitability for large-scale classification challenges such as applying convolutional neural networks to the 1000-class ImageNet dataset. These metrics, referred to as the "geometric accuracy," "decisiveness," and "robustness," are based on the generalized mean ($ρ$ equals 0, 1, and -2/3, respectively) of the classifier's self-reported and measured probabilities of correct classification. We also propose some minor clarifications to standardize the metric definitions. With these updates, we show some examples of calculating the metrics using deep convolutional neural networks (AlexNet and DenseNet) acting on large datasets (the German Traffic Sign Recognition Benchmark and ImageNet).