Spectral Logit Sculpting: Adaptive Low-Rank Logit Transformation for Controlled Text Generation
This addresses the need for more efficient and context-aware controlled text generation in LLMs, representing an incremental improvement over existing inference-time optimization techniques.
The paper tackled the problem of high computational overhead and ineffective use of historical token context in entropy-based inference methods for LLMs by proposing Spectral Logit Sculpting (SLS), a lightweight method that dynamically modulates token distributions, resulting in superior accuracy on mathematical, coding, and scientific reasoning benchmarks.
Entropy-based inference methods have gained traction for improving the reliability of Large Language Models (LLMs). However, many existing approaches, such as entropy minimization techniques, suffer from high computational overhead and fail to leverage historical token context effectively. To address these limitations, we propose Spectral Logit Sculpting (SLS), a lightweight inference-time optimization method that dynamically modulates token distributions using spectral and entropic properties of recent logits. SLS maintains a sliding buffer of top-K logits, performs on-the-fly Singular Value Decomposition (SVD) to identify dominant spectral directions, and adaptively rescales logits based on both entropy and logit gap statistics--only activating when uncertainty is high. Without updating any model parameters, SLS effectively sharpens the output distribution while preserving contextual consistency. Experimental results on multiple public benchmarks demonstrate that SLS consistently outperforms existing baseline methods, achieving superior accuracy in mathematical, coding, and scientific reasoning tasks.