IDEA: Interactive DoublE Attentions from Label Embedding for Text Classification
This work addresses text classification for researchers and practitioners by improving accuracy and stability through label text integration, representing an incremental advance in leveraging label semantics.
The paper tackles the problem of text classification by incorporating label text information, which is often ignored, and introduces the IDEA model using siamese BERT and interactive double attentions to capture text-label interactions, achieving significant performance improvements over state-of-the-art methods with more stable results.
Current text classification methods typically encode the text merely into embedding before a naive or complicated classifier, which ignores the suggestive information contained in the label text. As a matter of fact, humans classify documents primarily based on the semantic meaning of the subcategories. We propose a novel model structure via siamese BERT and interactive double attentions named IDEA ( Interactive DoublE Attentions) to capture the information exchange of text and label names. Interactive double attentions enable the model to exploit the inter-class and intra-class information from coarse to fine, which involves distinguishing among all labels and matching the semantical subclasses of ground truth labels. Our proposed method outperforms the state-of-the-art methods using label texts significantly with more stable results.