Learning to Decide with Just Enough: Information-Theoretic Context Summarization for CMDPs
This addresses the challenge of scalable and interpretable decision-making in context-rich, resource-constrained environments, representing a novel method for a known bottleneck.
The paper tackles the problem of inefficient decision-making in high-dimensional contextual Markov decision processes (CMDPs) by proposing an information-theoretic summarization approach using large language models (LLMs) to compress contextual inputs, resulting in improved reward, success rate, sample efficiency, and reduced latency and memory usage across benchmarks.
Contextual Markov Decision Processes (CMDPs) offer a framework for sequential decision-making under external signals, but existing methods often fail to generalize in high-dimensional or unstructured contexts, resulting in excessive computation and unstable performance. We propose an information-theoretic summarization approach that uses large language models (LLMs) to compress contextual inputs into low-dimensional, semantically rich summaries. These summaries augment states by preserving decision-critical cues while reducing redundancy. Building on the notion of approximate context sufficiency, we provide, to our knowledge, the first regret bounds and a latency-entropy trade-off characterization for CMDPs. Our analysis clarifies how informativeness impacts computational cost. Experiments across discrete, continuous, visual, and recommendation benchmarks show that our method outperforms raw-context and non-context baselines, improving reward, success rate, and sample efficiency, while reducing latency and memory usage. These findings demonstrate that LLM-based summarization offers a scalable and interpretable solution for efficient decision-making in context-rich, resource-constrained environments.