High Quality Diffusion Distillation on a Single GPU with Relative and Absolute Position Matching
This work addresses the resource limitations of researchers by providing a more accessible method for diffusion distillation, though it is incremental as it builds on existing techniques like phased consistency models.
The paper tackles the problem of high computational cost in diffusion distillation for text-to-image generation by introducing relative and absolute position matching (RAPM), which enables efficient training on a single GPU with a batch size of 1 while achieving comparable FID scores to state-of-the-art methods.
We introduce relative and absolute position matching (RAPM), a diffusion distillation method resulting in high quality generation that can be trained efficiently on a single GPU. Recent diffusion distillation research has achieved excellent results for high-resolution text-to-image generation with methods such as phased consistency models (PCM) and improved distribution matching distillation (DMD2). However, these methods generally require many GPUs (e.g.~8-64) and significant batchsizes (e.g.~128-2048) during training, resulting in memory and compute requirements that are beyond the resources of some researchers. RAPM provides effective single-GPU diffusion distillation training with a batchsize of 1. The new method attempts to mimic the sampling trajectories of the teacher model by matching the relative and absolute positions. The design of relative positions is inspired by PCM. Two discriminators are introduced accordingly in RAPM, one for matching relative positions and the other for absolute positions. Experimental results on StableDiffusion (SD) V1.5 and SDXL indicate that RAPM with 4 timesteps produces comparable FID scores as the best method with 1 timestep under very limited computational resources.