Supporting Mitosis Detection AI Training with Inter-Observer Eye-Gaze Consistencies
This addresses the annotation bottleneck in medical AI development, offering a cost-effective alternative for mitosis detection, though it is incremental as it builds on existing eye-tracking and CNN methods.
This study tackled the high cost and time of collecting doctor annotations for AI in pathology by using eye-tracking data from non-medical participants to generate labels based on inter-observer consistencies for mitosis detection. Results showed that CNNs trained with these eye-gaze labels performed nearly as well as those using ground truth annotations and significantly better than a baseline.
The expansion of artificial intelligence (AI) in pathology tasks has intensified the demand for doctors' annotations in AI development. However, collecting high-quality annotations from doctors is costly and time-consuming, creating a bottleneck in AI progress. This study investigates eye-tracking as a cost-effective technology to collect doctors' behavioral data for AI training with a focus on the pathology task of mitosis detection. One major challenge in using eye-gaze data is the low signal-to-noise ratio, which hinders the extraction of meaningful information. We tackled this by levering the properties of inter-observer eye-gaze consistencies and creating eye-gaze labels from consistent eye-fixations shared by a group of observers. Our study involved 14 non-medical participants, from whom we collected eye-gaze data and generated eye-gaze labels based on varying group sizes. We assessed the efficacy of such eye-gaze labels by training Convolutional Neural Networks (CNNs) and comparing their performance to those trained with ground truth annotations and a heuristic-based baseline. Results indicated that CNNs trained with our eye-gaze labels closely followed the performance of ground-truth-based CNNs, and significantly outperformed the baseline. Although primarily focused on mitosis, we envision that insights from this study can be generalized to other medical imaging tasks.