GazeSAM: What You See is What You Segment
This work addresses the need for faster and more intuitive segmentation in medical imaging for radiologists, though it is incremental as it combines existing technologies (eye-tracking and SAM) in a novel application.
This study tackled the problem of automating medical image segmentation by developing GazeSAM, a system that uses eye-tracking technology and the Segment Anything Model (SAM) to generate segmentation masks in real time based on radiologists' gaze, enhancing efficiency in clinical practice.
This study investigates the potential of eye-tracking technology and the Segment Anything Model (SAM) to design a collaborative human-computer interaction system that automates medical image segmentation. We present the \textbf{GazeSAM} system to enable radiologists to collect segmentation masks by simply looking at the region of interest during image diagnosis. The proposed system tracks radiologists' eye movement and utilizes the eye-gaze data as the input prompt for SAM, which automatically generates the segmentation mask in real time. This study is the first work to leverage the power of eye-tracking technology and SAM to enhance the efficiency of daily clinical practice. Moreover, eye-gaze data coupled with image and corresponding segmentation labels can be easily recorded for further advanced eye-tracking research. The code is available in \url{https://github.com/ukaukaaaa/GazeSAM}.