Informed Deep Abstaining Classifier: Investigating noise-robust training for diagnostic decision support systems
This work addresses the challenge of developing accurate diagnostic systems from noisy clinical data, which is incremental as it builds on prior noise-robust training methods.
The study tackled the problem of label noise in diagnostic decision support systems by proposing an Informed Deep Abstaining Classifier (IDAC) loss that incorporates noise level estimations, demonstrating enhanced noise robustness compared to existing methods on simulated and in-house noisy chest X-ray datasets.
Image-based diagnostic decision support systems (DDSS) utilizing deep learning have the potential to optimize clinical workflows. However, developing DDSS requires extensive datasets with expert annotations and is therefore costly. Leveraging report contents from radiological data bases with Natural Language Processing to annotate the corresponding image data promises to replace labor-intensive manual annotation. As mining "real world" databases can introduce label noise, noise-robust training losses are of great interest. However, current noise-robust losses do not consider noise estimations that can for example be derived based on the performance of the automatic label generator used. In this study, we expand the noise-robust Deep Abstaining Classifier (DAC) loss to an Informed Deep Abstaining Classifier (IDAC) loss by incorporating noise level estimations during training. Our findings demonstrate that IDAC enhances the noise robustness compared to DAC and several state-of-the-art loss functions. The results are obtained on various simulated noise levels using a public chest X-ray data set. These findings are reproduced on an in-house noisy data set, where labels were extracted from the clinical systems of the University Hospital Bonn by a text-based transformer. The IDAC can therefore be a valuable tool for researchers, companies or clinics aiming to develop accurate and reliable DDSS from routine clinical data.