Parameter-Efficient Fine-Tuning of 3D DDPM for MRI Image Generation Using Tensor Networks
This work addresses the problem of efficient fine-tuning for 3D MRI generation, which is incremental as it builds on existing DDPMs with a novel parameter-efficient approach.
The paper tackles parameter-efficient fine-tuning for 3D DDPMs in MRI image generation by proposing TenVOO, a method using tensor networks to reduce parameters, achieving state-of-the-art MS-SSIM performance with only 0.3% of trainable parameters.
We address the challenge of parameter-efficient fine-tuning (PEFT) for three-dimensional (3D) U-Net-based denoising diffusion probabilistic models (DDPMs) in magnetic resonance imaging (MRI) image generation. Despite its practical significance, research on parameter-efficient representations of 3D convolution operations remains limited. To bridge this gap, we propose Tensor Volumetric Operator (TenVOO), a novel PEFT method specifically designed for fine-tuning DDPMs with 3D convolutional backbones. Leveraging tensor network modeling, TenVOO represents 3D convolution kernels with lower-dimensional tensors, effectively capturing complex spatial dependencies during fine-tuning with few parameters. We evaluate TenVOO on three downstream brain MRI datasets-ADNI, PPMI, and BraTS2021-by fine-tuning a DDPM pretrained on 59,830 T1-weighted brain MRI scans from the UK Biobank. Our results demonstrate that TenVOO achieves state-of-the-art performance in multi-scale structural similarity index measure (MS-SSIM), outperforming existing approaches in capturing spatial dependencies while requiring only 0.3% of the trainable parameters of the original model. Our code is available at: https://github.com/xiaovhua/tenvoo