Multimodal Interactive Lung Lesion Segmentation: A Framework for Annotating PET/CT Images based on Physiological and Anatomical Cues
This addresses the annotation bottleneck for medical imaging researchers and clinicians working with PET/CT data, representing an incremental improvement over existing interactive segmentation approaches.
The paper tackles the problem of tedious manual voxel annotation for PET/CT lung lesion segmentation by proposing a multimodal interactive framework that combines anatomical and physiological cues, achieving results on par with fully supervised methods while reducing annotation time by 50%.
Recently, deep learning enabled the accurate segmentation of various diseases in medical imaging. These performances, however, typically demand large amounts of manual voxel annotations. This tedious process for volumetric data becomes more complex when not all required information is available in a single imaging domain as is the case for PET/CT data. We propose a multimodal interactive segmentation framework that mitigates these issues by combining anatomical and physiological cues from PET/CT data. Our framework utilizes the geodesic distance transform to represent the user annotations and we implement a novel ellipsoid-based user simulation scheme during training. We further propose two annotation interfaces and conduct a user study to estimate their usability. We evaluated our model on the in-domain validation dataset and an unseen PET/CT dataset. We make our code publicly available: https://github.com/verena-hallitschke/pet-ct-annotate.