Pixel-Level Residual Diffusion Transformer: Scalable 3D CT Volume Generation
For medical imaging researchers, PRDiT provides a scalable method for generating high-quality 3D CT volumes without autoencoder bottlenecks, though the improvement is incremental over existing diffusion-based approaches.
PRDiT generates high-resolution 3D CT volumes by separating low-frequency structures with a local denoiser and refining high-frequency residuals with a global diffusion transformer, outperforming HA-GAN, 3D LDM, and WDM-3D on LIDC-IDRI and RAD-ChestCT datasets with lower 3D FID, MMD, and Wasserstein distance scores.
Generating high-resolution 3D CT volumes with fine details remains challenging due to substantial computational demands and optimization difficulties inherent to existing generative models. In this paper, we propose the Pixel-Level Residual Diffusion Transformer (PRDiT), a scalable generative framework that synthesizes high-quality 3D medical volumes directly at voxel-level. PRDiT introduces a two-stage training architecture comprising 1) a local denoiser in the form of an MLP-based blind estimator operating on overlapping 3D patches to separate low-frequency structures efficiently, and 2) a global residual diffusion transformer employing memory-efficient attention to model and refine high-frequency residuals across entire volumes. This coarse-to-fine modeling strategy simplifies optimization, enhances training stability, and effectively preserves subtle structures without the limitations of an autoencoder bottleneck. Extensive experiments conducted on the LIDC-IDRI and RAD-ChestCT datasets demonstrate that PRDiT consistently outperforms state-of-the-art models, such as HA-GAN, 3D LDM and WDM-3D, achieving significantly lower 3D FID, MMD and Wasserstein distance scores.