Probabilistic Language Tries: A Unified Framework for Compression, Decision Policies, and Execution Reuse
This work provides a theoretical and practical framework that connects compression, decision-making, and computational reuse for sequence models, potentially impacting efficiency in large-scale inference and decision systems.
Probabilistic language tries (PLTs) unify compression, decision policies, and execution reuse by making explicit the prefix structure of generative models. The framework achieves O(n^2) transformer attention cost reduced to p_r * O(log N) + (1 - p_r) * O(n^2) via prior-guided caching, and is demonstrated across chess, web search, robotics, and LLM inference.
We introduce probabilistic language tries (PLTs), a unified representation that makes explicit the prefix structure implicitly defined by any generative model over sequences. By assigning to each outgoing edge the conditional probability of the corresponding token or action, a PLT simultaneously serves as: (i) an optimal lossless compressor via frequency-weighted interval encoding, generalizing arithmetic coding to model-conditioned distributions; (ii) a policy representation for sequential decision problems including games, search, and robotic control; and (iii) a memoization index that lets repeated inference queries be answered by structured retrieval rather than full model execution. The central technical result is a prior-guided caching theorem: under a stationary generative distribution, a PLT-guided cache achieves strictly lower expected inference cost than any empirical-frequency cache for all query counts below a threshold that grows with the concentration of the prior. This converts O(n^2) transformer attention cost into an expected cost of p_r * O(log N) + (1 - p_r) * O(n^2), where p_r is the prior-estimated reuse probability and N is the artifact store size. We further introduce a hybrid compression architecture decomposing any dataset into a PLT-covered majority and a sparse residual store, connecting arithmetic coding with Kolmogorov-style program representations and rate-distortion theory. We instantiate the framework across chess, web search, robotics, organizational workflows, and LLM inference, demonstrating that compression, decision making, and computational reuse are all derived from a single probability measure on sequence space.