Top-b: Entropic Regulation of Relative Probability Bands in Autoregressive Language Processes
This work addresses a domain-specific problem for users of autoregressive language models by offering an incremental improvement over existing decoding methods like Top-k and Top-p.
The paper tackled the problem of static decoding strategies in language generation causing suboptimal trade-offs between creativity and logical reasoning by introducing Top-b, a dynamic decoding method that adapts to the model's instantaneous entropy, resulting in reduced generation entropy and variance while maintaining competitive accuracy on GPQA and GSM8K benchmarks.
Probabilistic language generators are theoretically modeled as discrete stochastic processes, yet standard decoding strategies (Top-k, Top-p) impose static truncation rules that fail to accommodate the dynamic information density of natural language. This misalignment often forces a suboptimal trade-off: static bounds are either too restrictive for high-entropy creative generation or too permissive for low-entropy logical reasoning. In this work, we formalize the generation process as a trajectory through a relative probability manifold. We introduce Top-b (Adaptive Relative Band Sampling), a decoding strategy that regulates the candidate set via a dynamic bandwidth coefficient coupled strictly to the instantaneous Shannon entropy of the model's distribution. We provide a theoretical framework demonstrating that Top-b acts as a variance-minimizing operator on the tail distribution. Empirical validation on GPQA and GSM8K benchmarks indicates that Top-b significantly reduces generation entropy and inter-decoding variance while maintaining competitive reasoning accuracy, effectively approximating a self-regulating control system for autoregressive generation.