meval: A Statistical Toolbox for Fine-Grained Model Performance Analysis
This addresses the problem of identifying model failure modes in medical imaging for practitioners, though it is incremental as it provides a toolbox for existing statistical methods.
The authors tackled the challenge of statistically rigorous analysis of machine learning model performance across subgroups, presenting a toolbox that enables practitioners to assess subgroup performance disparities, as illustrated in case studies on skin lesion and chest X-ray classification datasets.
Analyzing machine learning model performance stratified by patient and recording properties is becoming the accepted norm and often yields crucial insights about important model failure modes. Performing such analyses in a statistically rigorous manner is non-trivial, however. Appropriate performance metrics must be selected that allow for valid comparisons between groups of different sample sizes and base rates; metric uncertainty must be determined and multiple comparisons be corrected for, in order to assess whether any observed differences may be purely due to chance; and in the case of intersectional analyses, mechanisms must be implemented to find the most `interesting' subgroups within combinatorially many subgroup combinations. We here present a statistical toolbox that addresses these challenges and enables practitioners to easily yet rigorously assess their models for potential subgroup performance disparities. While broadly applicable, the toolbox is specifically designed for medical imaging applications. The analyses provided by the toolbox are illustrated in two case studies, one in skin lesion malignancy classification on the ISIC2020 dataset and one in chest X-ray-based disease classification on the MIMIC-CXR dataset.