Large Language Models as Agents in Two-Player Games
It provides a novel theoretical framework for understanding and improving LLM training, though it is incremental as it builds on existing concepts without presenting new empirical results.
This position paper re-conceptualizes the training processes of large language models (LLMs) as agent learning in two-player games, drawing parallels from game theory and multi-agent systems to offer fresh insights into LLM development and alignment challenges.
By formally defining the training processes of large language models (LLMs), which usually encompasses pre-training, supervised fine-tuning, and reinforcement learning with human feedback, within a single and unified machine learning paradigm, we can glean pivotal insights for advancing LLM technologies. This position paper delineates the parallels between the training methods of LLMs and the strategies employed for the development of agents in two-player games, as studied in game theory, reinforcement learning, and multi-agent systems. We propose a re-conceptualization of LLM learning processes in terms of agent learning in language-based games. This framework unveils innovative perspectives on the successes and challenges in LLM development, offering a fresh understanding of addressing alignment issues among other strategic considerations. Furthermore, our two-player game approach sheds light on novel data preparation and machine learning techniques for training LLMs.