Playing log(N)-Questions over Sentences
This work addresses a challenge in multi-agent dialogue systems for natural language processing, but it appears incremental as it builds on existing game-based learning frameworks without claiming broad breakthroughs.
The authors tackled the problem of training agents to play a two-agent question-answering game over sentences, where a questioner must formulate discerning questions and an answerer responds, aiming for accurate final guesses. They found that achieving high game accuracy while producing meaningful questions presents a difficult trade-off, with experimental results showing specific performance metrics (e.g., accuracy rates) not provided in the abstract.
We propose a two-agent game wherein a questioner must be able to conjure discerning questions between sentences, incorporate responses from an answerer, and keep track of a hypothesis state. The questioner must be able to understand the information required to make its final guess, while also being able to reason over the game's text environment based on the answerer's responses. We experiment with an end-to-end model where both agents can learn simultaneously to play the game, showing that simultaneously achieving high game accuracy and producing meaningful questions can be a difficult trade-off.