Domain Lexical Knowledge-based Word Embedding Learning for Text Classification under Small Data
This work addresses performance issues in text classification under small data conditions for applications like emotion recognition and sentiment analysis, representing an incremental improvement.
The paper tackles the problem of BERT's insufficient performance in text classification tasks where keywords are critical, by enhancing word embeddings with domain-specific lexical knowledge to improve discriminative text representation. Experimental results on sentiment analysis, emotion recognition, and question answering tasks demonstrate the effectiveness of the proposed model.
Pre-trained language models such as BERT have been proved to be powerful in many natural language processing tasks. But in some text classification applications such as emotion recognition and sentiment analysis, BERT may not lead to satisfactory performance. This often happens in applications where keywords play critical roles in the prediction of class labels. Our investigation found that the root cause of the problem is that the context-based BERT embedding of the keywords may not be discriminative enough to produce discriminative text representation for classification. Motivated by this finding, we develop a method to enhance word embeddings using domain-specific lexical knowledge. The knowledge-based embedding enhancement model projects the BERT embedding into a new space where within-class similarity and between-class difference are maximized. To implement the knowledge-based word embedding enhancement model, we also develop a knowledge acquisition algorithm for automatically collecting lexical knowledge from online open sources. Experiment results on three classification tasks, including sentiment analysis, emotion recognition and question answering, have shown the effectiveness of our proposed word embedding enhancing model. The codes and datasets are in https://github.com/MidiyaZhu/KVWEFFER.