Behavioral Differences is the Key of Ad-hoc Team Cooperation in Multiplayer Games Hanabi
This work addresses the challenge of enabling AI agents to cooperate effectively with unfamiliar partners in multiplayer games, which is incremental as it analyzes and quantifies existing failure modes rather than introducing a new solution.
The paper tackled the problem of ad-hoc team cooperation in Hanabi, where agents fail to cooperate with unseen partners after reinforcement learning training, and found that larger behavioral differences between agents lead to more pronounced cooperation failures, with a correlation of -0.978 between behavioral differences and performance.
Ad-hoc team cooperation is the problem of cooperating with other players that have not been seen in the learning process. Recently, this problem has been considered in the context of Hanabi, which requires cooperation without explicit communication with the other players. While in self-play strategies cooperating on reinforcement learning (RL) process has shown success, there is the problem of failing to cooperate with other unseen agents after the initial learning is completed. In this paper, we categorize the results of ad-hoc team cooperation into Failure, Success, and Synergy and analyze the associated failures. First, we confirm that agents learning via RL converge to one strategy each, but not necessarily the same strategy and that these agents can deploy different strategies even though they utilize the same hyperparameters. Second, we confirm that the larger the behavioral difference, the more pronounced the failure of ad-hoc team cooperation, as demonstrated using hierarchical clustering and Pearson correlation. We confirm that such agents are grouped into distinctly different groups through hierarchical clustering, such that the correlation between behavioral differences and ad-hoc team performance is -0.978. Our results improve understanding of key factors to form successful ad-hoc team cooperation in multi-player games.