Learnability-Informed Fine-Tuning of Diffusion Language Models
For researchers working on diffusion language models, this work addresses the understudied problem of fine-tuning DLMs, showing that vanilla SFT is suboptimal and providing a principled alternative.
The paper identifies that standard supervised fine-tuning (SFT) hurts diffusion language models (DLMs) because it ignores token learnability. They propose LIFT, which aligns training with information availability across diffusion steps, achieving up to 3x relative gain on AIME'24 and AIME'25 reasoning benchmarks.
We aim to improve the reasoning capabilities of diffusion language models (DLMs). While SFT is a popular post-training recipe for autoregressive models, its use in DLMs faces challenges and can even hurt performance, though the underlying causes remain understudied. Our analysis reveals that vanilla SFT overlooks learnability, namely what and when tokens are learned. Specifically, rare tokens are difficult to learn when most of the input is masked, whereas it is straightforward and thus of little value to learn common tokens when most of the input is unmasked. Motivated by our analysis, we propose LIFT, an efficient SFT-based post-training algorithm for DLMs. LIFT learns easy tokens when most of the input is masked and hard tokens when more context is available, thus aligning the training with the information available at different diffusion time steps. Our results show that LIFT outperforms existing SFT baselines across six reasoning benchmarks, achieving up to a 3x relative gain on AIME'24 and AIME'25. Our code is publicly available at https://github.com/divelab/LIFT.