CooT: Learning to Coordinate In-Context with Coordination Transformers
This addresses coordination problems in dynamic multi-agent systems for AI researchers, offering a novel in-context learning approach that is more effective than existing methods.
The paper tackles the challenge of poor generalization and extensive fine-tuning in multi-agent coordination by proposing Coordination Transformers (CooT), which uses interaction histories to adapt to unseen partners, achieving stable, rapid in-context adaptation and outperforming baselines in tasks like Overcooked.
Effective coordination among artificial agents in dynamic and uncertain environments remains a significant challenge in multi-agent systems. Existing approaches, such as self-play and population-based methods, either generalize poorly to unseen partners or require impractically extensive fine-tuning. To overcome these limitations, we propose Coordination Transformers (\coot), a novel in-context coordination framework that uses recent interaction histories to rapidly adapt to unseen partners. Unlike prior approaches that primarily aim to diversify training partners, \coot explicitly focuses on adapting to new partner behaviors by predicting actions aligned with observed interactions. Trained on trajectories collected from diverse pairs of agents with complementary preferences, \coot quickly learns effective coordination strategies without explicit supervision or parameter updates. Across diverse coordination tasks in Overcooked, \coot consistently outperforms baselines including population-based approaches, gradient-based fine-tuning, and a Meta-RL-inspired contextual adaptation method. Notably, fine-tuning proves unstable and ineffective, while Meta-RL struggles to achieve reliable coordination. By contrast, \coot achieves stable, rapid in-context adaptation and is consistently ranked the most effective collaborator in human evaluations.