(G)I-DLE: Generative Inference via Distribution-preserving Logit Exclusion with KL Divergence Minimization for Constrained Decoding
This addresses constrained decoding for language models, offering a method to improve fluency and reduce variance in outputs, though it appears incremental as it builds on existing logit exclusion techniques.
The paper tackles the problem of constrained decoding in autoregressive language models by proposing (G)I-DLE, which uses KL divergence minimization to preserve probability distributions while excluding undesirable tokens, resulting in boosted mean evaluation scores and reduced output variance on Korean language tasks.
We propose (G)I-DLE, a new approach to constrained decoding that leverages KL divergence minimization to preserve the intrinsic conditional probability distribution of autoregressive language models while excluding undesirable tokens. Unlike conventional methods that naively set banned tokens' logits to $-\infty$, which can distort the conversion from raw logits to posterior probabilities and increase output variance, (G)I-DLE re-normalizes the allowed token probabilities to minimize such distortion. We validate our method on the K2-Eval dataset, specifically designed to assess Korean language fluency, logical reasoning, and cultural appropriateness. Experimental results on Qwen2.5 models (ranging from 1.5B to 14B) demonstrate that G-IDLE not only boosts mean evaluation scores but also substantially reduces the variance of output quality.