Hyper-Connected Transformer Network for Multi-Modality PET-CT Segmentation
This work addresses the need for accurate tumor segmentation in cancer diagnosis using PET-CT imaging, but it appears incremental as it builds on existing transformer and fusion techniques.
The authors tackled the problem of automatic tumor segmentation from multi-modality PET-CT images by proposing a hyper-connected transformer network, which achieved better segmentation accuracy compared to existing methods on two clinical datasets.
[18F]-Fluorodeoxyglucose (FDG) positron emission tomography - computed tomography (PET-CT) has become the imaging modality of choice for diagnosing many cancers. Co-learning complementary PET-CT imaging features is a fundamental requirement for automatic tumor segmentation and for developing computer aided cancer diagnosis systems. In this study, we propose a hyper-connected transformer (HCT) network that integrates a transformer network (TN) with a hyper connected fusion for multi-modality PET-CT images. The TN was leveraged for its ability to provide global dependencies in image feature learning, which was achieved by using image patch embeddings with a self-attention mechanism to capture image-wide contextual information. We extended the single-modality definition of TN with multiple TN based branches to separately extract image features. We also introduced a hyper connected fusion to fuse the contextual and complementary image features across multiple transformers in an iterative manner. Our results with two clinical datasets show that HCT achieved better performance in segmentation accuracy when compared to the existing methods.