The Swiss Army Knife for Image-to-Image Translation: Multi-Task Diffusion Models
This provides a flexible tool for medical imaging and photo editing by enabling multi-task guidance with a single model, though it is incremental as it builds on existing diffusion methods.
The paper tackles image-to-image translation by extending diffusion models to handle multiple tasks like regression and segmentation without retraining, achieving convincing results in simulating facial aging and brain tumor growth/inpainting.
Recently, diffusion models were applied to a wide range of image analysis tasks. We build on a method for image-to-image translation using denoising diffusion implicit models and include a regression problem and a segmentation problem for guiding the image generation to the desired output. The main advantage of our approach is that the guidance during the denoising process is done by an external gradient. Consequently, the diffusion model does not need to be retrained for the different tasks on the same dataset. We apply our method to simulate the aging process on facial photos using a regression task, as well as on a brain magnetic resonance (MR) imaging dataset for the simulation of brain tumor growth. Furthermore, we use a segmentation model to inpaint tumors at the desired location in healthy slices of brain MR images. We achieve convincing results for all problems.