Rethinking Machine Learning Model Evaluation in Pathology
This work addresses the challenge of rigorous ML evaluation for clinical decisions in pathology, but it is incremental as it builds on existing evaluation practices with domain-specific adaptations.
The paper tackles the problem of inadequate evaluation of machine learning models in pathology, proposing a set of practical guidelines to address issues like large noisy images and spurious correlations, aiming to bridge the gap between researchers and domain experts for wider adoption and improved patient outcomes.
Machine Learning has been applied to pathology images in research and clinical practice with promising outcomes. However, standard ML models often lack the rigorous evaluation required for clinical decisions. Machine learning techniques for natural images are ill-equipped to deal with pathology images that are significantly large and noisy, require expensive labeling, are hard to interpret, and are susceptible to spurious correlations. We propose a set of practical guidelines for ML evaluation in pathology that address the above concerns. The paper includes measures for setting up the evaluation framework, effectively dealing with variability in labels, and a recommended suite of tests to address issues related to domain shift, robustness, and confounding variables. We hope that the proposed framework will bridge the gap between ML researchers and domain experts, leading to wider adoption of ML techniques in pathology and improving patient outcomes.