Emergence of Communication in an Interactive World with Consistent Speakers
This work addresses the problem of enabling robust human-agent interaction through emergent communication, though it is incremental as it builds on prior methods for more challenging environments.
The paper tackles the challenge of training agents to communicate from raw pixels in complex environments where standard policy gradient fails, proposing a new model and training algorithm that stabilizes learning by producing consistent speakers early on. The results show substantial performance improvements and increased context-independence in communication compared to baselines.
Training agents to communicate with one another given task-based supervision only has attracted considerable attention recently, due to the growing interest in developing models for human-agent interaction. Prior work on the topic focused on simple environments, where training using policy gradient was feasible despite the non-stationarity of the agents during training. In this paper, we present a more challenging environment for testing the emergence of communication from raw pixels, where training using policy gradient fails. We propose a new model and training algorithm, that utilizes the structure of a learned representation space to produce more consistent speakers at the initial phases of training, which stabilizes learning. We empirically show that our algorithm substantially improves performance compared to policy gradient. We also propose a new alignment-based metric for measuring context-independence in emerged communication and find our method increases context-independence compared to policy gradient and other competitive baselines.