TRACE: Temporally Reliable Anatomically-Conditioned 3D CT Generation with Enhanced Efficiency
This work addresses the need for reliable and efficient 3D medical image generation in clinical practice, particularly for resource-limited regions, by improving anatomical fidelity and reducing computational costs.
The paper tackles the problem of generating 3D medical images for data augmentation and privacy, addressing issues like limited anatomical fidelity and high computational cost in current methods, and introduces TRACE, a framework that uses a 2D multimodal-conditioned diffusion approach to produce spatiotemporally aligned 3D CT volumes with enhanced efficiency.
3D medical image generation is essential for data augmentation and patient privacy, calling for reliable and efficient models suited for clinical practice. However, current methods suffer from limited anatomical fidelity, restricted axial length, and substantial computational cost, placing them beyond reach for regions with limited resources and infrastructure. We introduce TRACE, a framework that generates 3D medical images with spatiotemporal alignment using a 2D multimodal-conditioned diffusion approach. TRACE models sequential 2D slices as video frame pairs, combining segmentation priors and radiology reports for anatomical alignment, incorporating optical flow to sustain temporal coherence. During inference, an overlapping-frame strategy links frame pairs into a flexible length sequence, reconstructed into a spatiotemporally and anatomically aligned 3D volume. Experimental results demonstrate that TRACE effectively balances computational efficiency with preserving anatomical fidelity and spatiotemporal consistency. Code is available at: https://github.com/VinyehShaw/TRACE.