Exploring Unsupervised Pretraining and Sentence Structure Modelling for Winograd Schema Challenge
This work addresses the problem of common sense representation and reasoning in AI, providing incremental improvements for researchers in natural language understanding.
The paper tackled the Winograd Schema Challenge, an AI-hard problem for common sense reasoning, by jointly modeling sentence structures and using pretraining models with fine-tuning, achieving a new state-of-the-art accuracy of 71.1%.
Winograd Schema Challenge (WSC) was proposed as an AI-hard problem in testing computers' intelligence on common sense representation and reasoning. This paper presents the new state-of-theart on WSC, achieving an accuracy of 71.1%. We demonstrate that the leading performance benefits from jointly modelling sentence structures, utilizing knowledge learned from cutting-edge pretraining models, and performing fine-tuning. We conduct detailed analyses, showing that fine-tuning is critical for achieving the performance, but it helps more on the simpler associative problems. Modelling sentence dependency structures, however, consistently helps on the harder non-associative subset of WSC. Analysis also shows that larger fine-tuning datasets yield better performances, suggesting the potential benefit of future work on annotating more Winograd schema sentences.