Impact of individual rater style on deep learning uncertainty in medical imaging segmentation
This addresses the issue of improving segmentation reliability in medical imaging by accounting for rater variability, though it is incremental as it builds on prior work on inter-rater variability.
This study tackled the problem of how individual rater style, quantified as bias and consistency, affects deep learning uncertainty in medical image segmentation, finding strong correlations (R² = 0.60 and 0.93) between rater bias and model uncertainty and showing that multi-center consensuses reduce uncertainty more effectively than single-center ones.
While multiple studies have explored the relation between inter-rater variability and deep learning model uncertainty in medical segmentation tasks, little is known about the impact of individual rater style. This study quantifies rater style in the form of bias and consistency and explores their impacts when used to train deep learning models. Two multi-rater public datasets were used, consisting of brain multiple sclerosis lesion and spinal cord grey matter segmentation. On both datasets, results show a correlation ($R^2 = 0.60$ and $0.93$) between rater bias and deep learning uncertainty. The impact of label fusion between raters' annotations on this relationship is also explored, and we show that multi-center consensuses are more effective than single-center consensuses to reduce uncertainty, since rater style is mostly center-specific.