Improved Cooperation by Exploiting a Common Signal
This work addresses the challenge of achieving cooperation and sustainable resource management for decentralized artificial agents in large-scale, low-observability common-pool resource environments, which is an incremental step towards more robust multi-agent systems.
This paper investigates how artificial agents can achieve cooperation in common-pool resource problems by exploiting a common signal, inspired by human conventions. By introducing an arbitrary common signal, agents achieved sustainable harvesting strategies, improving social welfare by 258% on average (up to 3306%), extending the range of sustainable environmental parameters by 46% (up to 300%), and increasing convergence speed in low abundance settings by 13% (up to 53%).
Can artificial agents benefit from human conventions? Human societies manage to successfully self-organize and resolve the tragedy of the commons in common-pool resources, in spite of the bleak prediction of non-cooperative game theory. On top of that, real-world problems are inherently large-scale and of low observability. One key concept that facilitates human coordination in such settings is the use of conventions. Inspired by human behavior, we investigate the learning dynamics and emergence of temporal conventions, focusing on common-pool resources. Extra emphasis was given in designing a realistic evaluation setting: (a) environment dynamics are modeled on real-world fisheries, (b) we assume decentralized learning, where agents can observe only their own history, and (c) we run large-scale simulations (up to 64 agents). Uncoupled policies and low observability make cooperation hard to achieve; as the number of agents grow, the probability of taking a correct gradient direction decreases exponentially. By introducing an arbitrary common signal (e.g., date, time, or any periodic set of numbers) as a means to couple the learning process, we show that temporal conventions can emerge and agents reach sustainable harvesting strategies. The introduction of the signal consistently improves the social welfare (by 258% on average, up to 3306%), the range of environmental parameters where sustainability can be achieved (by 46% on average, up to 300%), and the convergence speed in low abundance settings (by 13% on average, up to 53%).