Beyond Winning and Losing: Modeling Human Motivations and Behaviors Using Inverse Reinforcement Learning
This work addresses the need to understand diverse human motivations in interactive environments like games, which is incremental as it extends existing RL methods to incorporate multifaceted motivations.
The paper tackles the problem of modeling complex human motivations in gameplay beyond just winning, using a novel inverse reinforcement learning method called Multi-Motivation Behavior Modeling (MMBM) on the World of Warcraft dataset with over 70,000 users, revealing significant differences in value structures among player groups and predicting diverse behaviors.
In recent years, reinforcement learning (RL) methods have been applied to model gameplay with great success, achieving super-human performance in various environments, such as Atari, Go, and Poker. However, those studies mostly focus on winning the game and have largely ignored the rich and complex human motivations, which are essential for understanding different players' diverse behaviors. In this paper, we present a novel method called Multi-Motivation Behavior Modeling (MMBM) that takes the multifaceted human motivations into consideration and models the underlying value structure of the players using inverse RL. Our approach does not require the access to the dynamic of the system, making it feasible to model complex interactive environments such as massively multiplayer online games. MMBM is tested on the World of Warcraft Avatar History dataset, which recorded over 70,000 users' gameplay spanning three years period. Our model reveals the significant difference of value structures among different player groups. Using the results of motivation modeling, we also predict and explain their diverse gameplay behaviors and provide a quantitative assessment of how the redesign of the game environment impacts players' behaviors.