Revealing Human Internal Attention Patterns from Gameplay Analysis for Reinforcement Learning
This work addresses the problem of understanding human-agent attention differences for researchers in reinforcement learning and human-computer interaction, though it is incremental as it builds on existing offline attention techniques.
The study tackled the problem of revealing human internal attention patterns from gameplay data by proposing contextualized, task-relevant attention networks, which generated attention maps from human and RL agent gameplay in Atari environments. The results showed that human attention maps were more sparse than agent maps and aligned better with eye-tracking data, and using these maps to guide RL agents led to slightly improved and more stable learning.
This study introduces a novel method for revealing human internal attention patterns from gameplay data alone, leveraging offline attention techniques from reinforcement learning (RL). We propose contextualized, task-relevant (CTR) attention networks, which generate attention maps from both human and RL agent gameplay in Atari environments. To evaluate whether the human CTR maps reveal internal attention, we validate our model by quantitative and qualitative comparison to the agent maps as well as to a temporally integrated overt attention (TIOA) model based on human eye-tracking data. Our results show that human CTR maps are more sparse than the agent ones and align better with the TIOA maps. Following a qualitative visual comparison we conclude that they likely capture patterns of internal attention. As a further application, we use these maps to guide RL agents, finding that human internal attention-guided agents achieve slightly improved and more stable learning compared to baselines. This work advances the understanding of human-agent attention differences and provides a new approach for extracting and validating internal attention from behavioral data.