TreeMix: Compositional Constituency-based Data Augmentation for Natural Language Understanding
This addresses data scarcity and over-fitting in NLP tasks, offering a novel approach to enhance model robustness through compositionality, though it is incremental in building on existing data augmentation techniques.
The paper tackles over-fitting in natural language understanding by proposing TreeMix, a compositional data augmentation method that uses constituency parsing and Mixup to generate diverse sentences, resulting in outperforming state-of-the-art methods on text classification and SCAN benchmarks.
Data augmentation is an effective approach to tackle over-fitting. Many previous works have proposed different data augmentations strategies for NLP, such as noise injection, word replacement, back-translation etc. Though effective, they missed one important characteristic of language--compositionality, meaning of a complex expression is built from its sub-parts. Motivated by this, we propose a compositional data augmentation approach for natural language understanding called TreeMix. Specifically, TreeMix leverages constituency parsing tree to decompose sentences into constituent sub-structures and the Mixup data augmentation technique to recombine them to generate new sentences. Compared with previous approaches, TreeMix introduces greater diversity to the samples generated and encourages models to learn compositionality of NLP data. Extensive experiments on text classification and SCAN demonstrate that TreeMix outperforms current state-of-the-art data augmentation methods.