Deriving Commonsense Inference Tasks from Interactive Fictions
This work addresses the need for better commonsense reasoning benchmarks in AI, though it is incremental as it builds on prior datasets.
The authors tackled the problem of commonsense reasoning in AI by creating a new dataset derived from interactive fiction gameplay, which human experts can solve but existing machine reading models struggle with, showing a performance gap of over 30%.
Commonsense reasoning simulates the human ability to make presumptions about our physical world, and it is an indispensable cornerstone in building general AI systems. We propose a new commonsense reasoning dataset based on human's interactive fiction game playings as human players demonstrate plentiful and diverse commonsense reasoning. The new dataset mitigates several limitations of the prior art. Experiments show that our task is solvable to human experts with sufficient commonsense knowledge but poses challenges to existing machine reading models, with a big performance gap of more than 30%.