The State-Action-Reward-State-Action Algorithm in Spatial Prisoner's Dilemma Game
This work addresses the challenge of modeling cooperation in social sciences and biology, but it is incremental as it applies an existing RL method to a known game theory context.
The study tackled the problem of understanding cooperative behavior in evolutionary game theory by applying the SARSA reinforcement learning algorithm to agents in a spatial prisoner's dilemma game, resulting in analysis of cooperation rates and agent distributions.
Cooperative behavior is prevalent in both human society and nature. Understanding the emergence and maintenance of cooperation among self-interested individuals remains a significant challenge in evolutionary biology and social sciences. Reinforcement learning (RL) provides a suitable framework for studying evolutionary game theory as it can adapt to environmental changes and maximize expected benefits. In this study, we employ the State-Action-Reward-State-Action (SARSA) algorithm as the decision-making mechanism for individuals in evolutionary game theory. Initially, we apply SARSA to imitation learning, where agents select neighbors to imitate based on rewards. This approach allows us to observe behavioral changes in agents without independent decision-making abilities. Subsequently, SARSA is utilized for primary agents to independently choose cooperation or betrayal with their neighbors. We evaluate the impact of SARSA on cooperation rates by analyzing variations in rewards and the distribution of cooperators and defectors within the network.