Not Enough Data? Deep Learning to the Rescue!
This addresses data scarcity in text classification, offering a practical solution for domains with limited labeled data, though it is incremental as it builds on existing language modeling and text generation capabilities.
The paper tackles the problem of scarce labeled data in text classification by proposing a novel data augmentation method called LAMBADA, which uses a fine-tuned language model to generate and filter new labeled sentences, resulting in improved classifier performance on various datasets and outperforming state-of-the-art techniques.
Based on recent advances in natural language modeling and those in text generation capabilities, we propose a novel data augmentation method for text classification tasks. We use a powerful pre-trained neural network model to artificially synthesize new labeled data for supervised learning. We mainly focus on cases with scarce labeled data. Our method, referred to as language-model-based data augmentation (LAMBADA), involves fine-tuning a state-of-the-art language generator to a specific task through an initial training phase on the existing (usually small) labeled data. Using the fine-tuned model and given a class label, new sentences for the class are generated. Our process then filters these new sentences by using a classifier trained on the original data. In a series of experiments, we show that LAMBADA improves classifiers' performance on a variety of datasets. Moreover, LAMBADA significantly improves upon the state-of-the-art techniques for data augmentation, specifically those applicable to text classification tasks with little data.