Towards Zero-shot Commonsense Reasoning with Self-supervised Refinement of Language Models
This addresses the problem of commonsense reasoning in AI for applications like natural language understanding, though it appears incremental as it builds on existing models.
The paper tackled zero-shot commonsense reasoning by refining a pre-trained language model with self-supervised learning, achieving viability on multiple benchmarks without annotated data.
Can we get existing language models and refine them for zero-shot commonsense reasoning? This paper presents an initial study exploring the feasibility of zero-shot commonsense reasoning for the Winograd Schema Challenge by formulating the task as self-supervised refinement of a pre-trained language model. In contrast to previous studies that rely on fine-tuning annotated datasets, we seek to boost conceptualization via loss landscape refinement. To this end, we propose a novel self-supervised learning approach that refines the language model utilizing a set of linguistic perturbations of similar concept relationships. Empirical analysis of our conceptually simple framework demonstrates the viability of zero-shot commonsense reasoning on multiple benchmarks.