DLM: Unified Decision Language Models for Offline Multi-Agent Sequential Decision Making
For offline multi-agent reinforcement learning, DLM provides a scalable and reusable policy that generalizes across heterogeneous observations and actions, addressing the limitation of fixed formats in existing methods.
The paper proposes Decision Language Model (DLM), a unified framework that formulates multi-agent decision making as dialogue-style sequence prediction, achieving superior performance over offline MARL baselines and LLM-based methods while demonstrating strong zero-shot generalization to unseen tasks.
Building scalable and reusable multi-agent decision policies from offline datasets remains a challenge in offline multi-agent reinforcement learning (MARL), as existing methods often rely on fixed observation formats and action spaces that limit generalization. In contrast, large language models (LLMs) offer a flexible modeling interface that can naturally accommodate heterogeneous observations and actions. Motivated by this, we propose the Decision Language Model (DLM), which formulates multi-agent decision making as a dialogue-style sequence prediction problem under the centralized training with decentralized execution paradigm. DLM is trained in two stages: a supervised fine-tuning phase, which leverages dialogue-style datasets for centralized training with inter-agent context and generates executable actions from offline trajectories, followed by a group relative policy optimization phase to enhance robustness to out-of-distribution actions through lightweight reward functions. Experiments on multiple benchmarks show that a unified DLM outperforms strong offline MARL baselines and LLM-based conversational decision-making methods, while demonstrating strong zero-shot generalization to unseen scenarios across tasks.