Clearing noisy annotations for computed tomography imaging
This addresses annotation noise in medical imaging for researchers and practitioners, but it is incremental as it builds on existing methods for handling multiple annotators.
The paper tackles the problem of noisy annotations in computed tomography (CT) imaging segmentation by proposing a clearing algorithm that assigns confidence levels to annotators and nodules, then merges annotations. The result is improved annotation quality, though no concrete numbers are provided.
One of the problems on the way to successful implementation of neural networks is the quality of annotation. For instance, different annotators can annotate images in a different way and very often their decisions do not match exactly and in extreme cases are even mutually exclusive which results in noisy annotations and, consequently, inaccurate predictions. To avoid that problem in the task of computed tomography (CT) imaging segmentation we propose a clearing algorithm for annotations. It consists of 3 stages: - annotators scoring, which assigns a higher confidence level to better annotators; - nodules scoring, which assigns a higher confidence level to nodules confirmed by good annotators; - nodules merging, which aggregates annotations according to nodules confidence. In general, the algorithm can be applied to many different tasks (namely, binary and multi-class semantic segmentation, and also with trivial adjustments to classification and regression) where there are several annotators labeling each image.