DiscoSense: Commonsense Reasoning with Discourse Connectives
This provides a new benchmark for evaluating commonsense reasoning systems, which is incremental as it builds on existing adversarial filtering techniques.
The authors introduced DiscoSense, a benchmark for commonsense reasoning that focuses on understanding discourse connectives, and generated distractors using Conditional Adversarial Filtering, showing that state-of-the-art pre-trained language models perform poorly on it.
We present DiscoSense, a benchmark for commonsense reasoning via understanding a wide variety of discourse connectives. We generate compelling distractors in DiscoSense using Conditional Adversarial Filtering, an extension of Adversarial Filtering that employs conditional generation. We show that state-of-the-art pre-trained language models struggle to perform well on DiscoSense, which makes this dataset ideal for evaluating next-generation commonsense reasoning systems.