Markovian Generation Chains in Large Language Models
This provides insights into the dynamics of iterative LLM inference, which is relevant for multi-agent LLM systems, but it is incremental as it builds on existing understanding of LLM behavior.
The paper tackles the problem of how texts evolve when repeatedly processed by large language models (LLMs) by defining this as Markovian generation chains, finding that outputs either converge to a small recurrent set or produce novel sentences over a finite horizon, with sentence diversity affected by factors like temperature and initial input.
The widespread use of large language models (LLMs) raises an important question: how do texts evolve when they are repeatedly processed by LLMs? In this paper, we define this iterative inference process as Markovian generation chains, where each step takes a specific prompt template and the previous output as input, without including any prior memory. In iterative rephrasing and round-trip translation experiments, the output either converges to a small recurrent set or continues to produce novel sentences over a finite horizon. Through sentence-level Markov chain modeling and analysis of simulated data, we show that iterative process can either increase or reduce sentence diversity depending on factors such as the temperature parameter and the initial input sentence. These results offer valuable insights into the dynamics of iterative LLM inference and their implications for multi-agent LLM systems.