Creation and Validation of a Chest X-Ray Dataset with Eye-tracking and Report Dictation for AI Development
This dataset addresses the need for multimodal data to advance explainable AI and automated radiology tools, benefiting researchers in disease classification and report generation, though it is incremental as it adds new data types to existing efforts.
The researchers created a dataset of 1,083 Chest X-Ray images with aligned eye-tracking, audio dictation, and transcribed reports to support AI development, demonstrating its utility through deep learning experiments that use eye gaze attention maps.
We developed a rich dataset of Chest X-Ray (CXR) images to assist investigators in artificial intelligence. The data were collected using an eye tracking system while a radiologist reviewed and reported on 1,083 CXR images. The dataset contains the following aligned data: CXR image, transcribed radiology report text, radiologist's dictation audio and eye gaze coordinates data. We hope this dataset can contribute to various areas of research particularly towards explainable and multimodal deep learning / machine learning methods. Furthermore, investigators in disease classification and localization, automated radiology report generation, and human-machine interaction can benefit from these data. We report deep learning experiments that utilize the attention maps produced by eye gaze dataset to show the potential utility of this data.