Multi-stage intermediate fusion for multimodal learning to classify non-small cell lung cancer subtypes from CT and PET
This work addresses the need for accurate, non-invasive diagnostic tools in precision medicine for lung cancer patients, representing a domain-specific incremental improvement.
The study tackled the problem of non-invasive classification of non-small cell lung cancer subtypes from CT and PET images by proposing a multi-stage intermediate fusion method, achieving an accuracy of 0.724 and AUC of 0.681, outperforming existing approaches.
Accurate classification of histological subtypes of non-small cell lung cancer (NSCLC) is essential in the era of precision medicine, yet current invasive techniques are not always feasible and may lead to clinical complications. This study presents a multi-stage intermediate fusion approach to classify NSCLC subtypes from CT and PET images. Our method integrates the two modalities at different stages of feature extraction, using voxel-wise fusion to exploit complementary information across varying abstraction levels while preserving spatial correlations. We compare our method against unimodal approaches using only CT or PET images to demonstrate the benefits of modality fusion, and further benchmark it against early and late fusion techniques to highlight the advantages of intermediate fusion during feature extraction. Additionally, we compare our model with the only existing intermediate fusion method for histological subtype classification using PET/CT images. Our results demonstrate that the proposed method outperforms all alternatives across key metrics, with an accuracy and AUC equal to 0.724 and 0.681, respectively. This non-invasive approach has the potential to significantly improve diagnostic accuracy, facilitate more informed treatment decisions, and advance personalized care in lung cancer management.