ConventionPlay: Capability-Limited Training for Robust Ad-Hoc Collaboration
For multi-agent systems requiring ad-hoc collaboration, this work addresses the challenge of steering teams toward effective joint strategies when partners can follow multiple conventions.
ConventionPlay trains an RL agent to probe and adapt to partners with varied capability limits in ad-hoc coordination tasks, achieving superior coordination efficiency especially when conventions have differentiated payoffs.
Ad-hoc collaboration often relies on identifying and adhering to shared conventions. However, when partners can follow multiple conventions, agents must do more than simply adapt; they must actively steer the team toward the most effective joint strategy. We present ConventionPlay, a reinforcement learning-based approach that extends cognitive hierarchies to include a diverse population of adaptive followers. By training against partners with varied capability limits, our agent learns to probe its partner's repertoire, leading the team when possible and following when necessary. Our results in canonical coordination tasks show that ConventionPlay achieves superior coordination efficiency, particularly in settings where conventions have differentiated payoffs.