DSFT: Inspiring Diffusion Large Language Models to Comprehend Mathematical and Logical Patterns
This work addresses a specific bottleneck in dLLMs for tasks requiring numerical and logical precision, offering an incremental but effective solution that can be combined with other training methods.
The paper tackles the challenge of diffusion large language models (dLLMs) struggling with mathematical and logical tasks by proposing DSFT, a diffusion SFT strategy that adjusts masking and loss functions, achieving improvements of 5-10% on mathematical problems and about 2% on logical problems.
Diffusion large language models (dLLMs) have emerged as a new architecture following auto regressive models. Their denoising process offers a powerful generative advantage, but they present significant challenges in learning and understanding numerically sensitive mathematical and order-sensitive logical tasks. Current training methods, including pre-training, fine-tuning, and reinforcement learning, focus primarily on improving general knowledge retention and reasoning abilities, but lack a comprehensive understanding of mathematical and logical patterns. We propose DSFT, a simple yet effective Diffusion SFT strategy, by adjusting the masking strategy and loss function, guiding models to understand mathematical and logical patterns. This strategy can be flexibly combined with pre-training, reinforcement learning, and other training methods. Validated on models such as LLaDA and Dream series, we prove that DSFT on small-scale data can achieve improvements of 5-10% and approximately 2% on mathematical and logical problems, respectively. This inspiring masking approach offers insights for future learning of specific patterns, which can be easily and efficiently combined with other training methods and applied to various dLLMs. Our code is publicly available at https://anonymous.4open.science/r/DSFT-0FFB/