Heavy-tailed Representations, Text Polarity Classification & Data Augmentation
This work addresses text classification and augmentation for natural language processing, offering a novel approach to handle distributional extremes, but it is incremental as it builds on existing embedding and extreme value theory frameworks.
The paper tackled the problem of learning text representations with heavy-tailed distributions to analyze extreme data points, resulting in a classifier that outperforms baselines and a data augmentation method that generates meaningful sentences with controllable sentiment.
The dominant approaches to text representation in natural language rely on learning embeddings on massive corpora which have convenient properties such as compositionality and distance preservation. In this paper, we develop a novel method to learn a heavy-tailed embedding with desirable regularity properties regarding the distributional tails, which allows to analyze the points far away from the distribution bulk using the framework of multivariate extreme value theory. In particular, a classifier dedicated to the tails of the proposed embedding is obtained which performance outperforms the baseline. This classifier exhibits a scale invariance property which we leverage by introducing a novel text generation method for label preserving dataset augmentation. Numerical experiments on synthetic and real text data demonstrate the relevance of the proposed framework and confirm that this method generates meaningful sentences with controllable attribute, e.g. positive or negative sentiment.