Riposte! A Large Corpus of Counter-Arguments
This addresses a gap for researchers in NLP and education by providing a resource for automated counter-argument generation, though it is incremental as it builds on existing work on constructive feedback.
The paper tackles the lack of a large-scale corpus for counter-arguments to improve critical thinking by creating Riposte!, a dataset of over 18k counter-arguments produced by crowdworkers to identify fallacies in micro-level arguments.
Constructive feedback is an effective method for improving critical thinking skills. Counter-arguments (CAs), one form of constructive feedback, have been proven to be useful for critical thinking skills. However, little work has been done for constructing a large-scale corpus of them which can drive research on automatic generation of CAs for fallacious micro-level arguments (i.e. a single claim and premise pair). In this work, we cast providing constructive feedback as a natural language processing task and create Riposte!, a corpus of CAs, towards this goal. Produced by crowdworkers, Riposte! contains over 18k CAs. We instruct workers to first identify common fallacy types and produce a CA which identifies the fallacy. We analyze how workers create CAs and construct a baseline model based on our analysis.