Emergence of Language with Multi-agent Games: Learning to Communicate with Sequences of Symbols
This work addresses the challenge of enabling AI agents to learn language-like communication without explicit supervision, which is incremental as it builds on prior referential game research by enforcing discrete symbol sequences.
The paper tackles the problem of developing communication protocols from scratch in multi-agent referential games, requiring messages to be sequences of discrete symbols. It finds that using a differentiable relaxation method converges faster and yields more effective protocols than reinforcement learning, with the induced protocols showing compositionality and variability similar to natural languages.
Learning to communicate through interaction, rather than relying on explicit supervision, is often considered a prerequisite for developing a general AI. We study a setting where two agents engage in playing a referential game and, from scratch, develop a communication protocol necessary to succeed in this game. Unlike previous work, we require that messages they exchange, both at train and test time, are in the form of a language (i.e. sequences of discrete symbols). We compare a reinforcement learning approach and one using a differentiable relaxation (straight-through Gumbel-softmax estimator) and observe that the latter is much faster to converge and it results in more effective protocols. Interestingly, we also observe that the protocol we induce by optimizing the communication success exhibits a degree of compositionality and variability (i.e. the same information can be phrased in different ways), both properties characteristic of natural languages. As the ultimate goal is to ensure that communication is accomplished in natural language, we also perform experiments where we inject prior information about natural language into our model and study properties of the resulting protocol.