Robert Seifert

Alejandro Lopez-Montes, Robert Seifert, Astrid Delker et al.

In this work we explored the use of patient specific reinforced learning to generate 3D activity maps from two 2D planar images (anterior and posterior). The solution of this problem remains unachievable using conventional methodologies and is of particular interest for dosimetry in nuclear medicine where approaches for post-therapy distribution of radiopharmaceuticals such as 177Lu-PSMA are typically done via either expensive and long 3D SPECT acquisitions or fast, yet only 2D, planar scintigraphy. Being able to generate 3D activity maps from planar scintigraphy opens the gate for new dosimetry applications removing the need for SPECT and facilitating multi-time point dosimetry studies. Our solution comprises the generation of a patient specific dataset with possible 3D uptake maps of the radiopharmaceuticals withing the anatomy of the individual followed by an AI approach (we explored both the use of 3DUnet and diffusion models) able to generate 3D activity maps from 2D planar images. We have validated our method both in simulation and real planar acquisitions. We observed enhanced results using patient specific reinforcement learning (~20% reduction on MAE and ~5% increase in SSIM) and better organ delineation and patient anatomy especially when combining diffusion models with patient specific training yielding a SSIM=0.89 compared to the ground truth for simulations and 0.73 when compared to a SPECT acquisition performed half an hour after the planar. We believe that our methodology can set a change of paradigm for nuclear medicine dosimetry allowing for 3D quantification using only planar scintigraphy without the need of expensive and time-consuming SPECT leveraging the pre-therapy information of the patients.

Yujin Oh, Robert Seifert, Yihan Cao et al.

In oncology, Positron Emission Tomography-Computed Tomography (PET/CT) is widely used in cancer diagnosis, staging, and treatment monitoring, as it combines anatomical details from CT with functional metabolic activity and molecular marker expression information from PET. However, existing artificial intelligence-driven PET/CT analyses rely predominantly on task-specific models trained from scratch or on limited datasets, limiting their generalizability and robustness. To address this, we propose a foundation model approach specifically designed for multimodal PET/CT imaging. We introduce the Cross-Fraternal Twin Masked Autoencoder (FratMAE), a novel framework that effectively integrates whole-body anatomical and functional or molecular information. FratMAE employs separate Vision Transformer (ViT) encoders for PET and CT scans, along with cross-attention decoders that enable synergistic interactions between modalities during masked autoencoder training. Additionally, it incorporates textual metadata to enhance PET representation learning. By pre-training on PET/CT datasets, FratMAE captures intricate cross-modal relationships and global uptake patterns, achieving superior performance on downstream tasks and demonstrating its potential as a generalizable foundation model.

Alejandro Lopez-Montes, Fereshteh Yousefirizi, Yizhou Chen et al.

KEY WORDS: Artificial Intelligence (AI), Theranostics, Dosimetry, Radiopharmaceutical Therapy (RPT), Patient-friendly dosimetry KEY POINTS - The rapid evolution of radiopharmaceutical therapy (RPT) highlights the growing need for personalized and patient-centered dosimetry. - Artificial Intelligence (AI) offers solutions to the key limitations in current dosimetry calculations. - The main advances on AI for simplified dosimetry toward patient-friendly RPT are reviewed. - Future directions on the role of AI in RPT dosimetry are discussed.