Text Classification with Few Examples using Controlled Generalization
This addresses the challenge of data scarcity in text classification for applications with many classes or related tasks, offering an incremental improvement over existing methods.
The paper tackles the problem of text classification with limited training data by developing a method that uses sparse pre-trained representations to create task-specific semantic vectors, achieving state-of-the-art performance in low-data scenarios and remaining competitive with full datasets.
Training data for text classification is often limited in practice, especially for applications with many output classes or involving many related classification problems. This means classifiers must generalize from limited evidence, but the manner and extent of generalization is task dependent. Current practice primarily relies on pre-trained word embeddings to map words unseen in training to similar seen ones. Unfortunately, this squishes many components of meaning into highly restricted capacity. Our alternative begins with sparse pre-trained representations derived from unlabeled parsed corpora; based on the available training data, we select features that offers the relevant generalizations. This produces task-specific semantic vectors; here, we show that a feed-forward network over these vectors is especially effective in low-data scenarios, compared to existing state-of-the-art methods. By further pairing this network with a convolutional neural network, we keep this edge in low data scenarios and remain competitive when using full training sets.