Generative Data Augmentation for Commonsense Reasoning
This addresses the need for more efficient and robust training data in low-resource commonsense reasoning tasks, though it is incremental as it builds on existing data augmentation and language model techniques.
The paper tackles the problem of expensive and artifact-prone human annotation in commonsense reasoning by proposing G-DAUG^C, a generative data augmentation method that uses pretrained language models to generate and select synthetic examples, resulting in new state-of-the-art performance on benchmarks like WinoGrande, CODAH, and CommonsenseQA and improved out-of-distribution generalization.
Recent advances in commonsense reasoning depend on large-scale human-annotated training data to achieve peak performance. However, manual curation of training examples is expensive and has been shown to introduce annotation artifacts that neural models can readily exploit and overfit on. We investigate G-DAUG^C, a novel generative data augmentation method that aims to achieve more accurate and robust learning in the low-resource setting. Our approach generates synthetic examples using pretrained language models, and selects the most informative and diverse set of examples for data augmentation. In experiments with multiple commonsense reasoning benchmarks, G-DAUG^C consistently outperforms existing data augmentation methods based on back-translation, and establishes a new state-of-the-art on WinoGrande, CODAH, and CommonsenseQA. Further, in addition to improvements in in-distribution accuracy, G-DAUG^C-augmented training also enhances out-of-distribution generalization, showing greater robustness against adversarial or perturbed examples. Our analysis demonstrates that G-DAUG^C produces a diverse set of fluent training examples, and that its selection and training approaches are important for performance. Our findings encourage future research toward generative data augmentation to enhance both in-distribution learning and out-of-distribution generalization.