HPT: Hierarchy-aware Prompt Tuning for Hierarchical Text Classification
This work addresses hierarchical text classification, a complex multi-label problem, by enhancing the use of pretrained language models, offering incremental improvements for researchers and practitioners in natural language processing.
The paper tackled the challenge of hierarchical text classification by proposing HPT, a hierarchy-aware prompt tuning method that bridges the gap between classification tasks and pretrained language models, achieving state-of-the-art performance on three popular datasets with improvements in handling imbalance and low-resource situations.
Hierarchical text classification (HTC) is a challenging subtask of multi-label classification due to its complex label hierarchy. Recently, the pretrained language models (PLM)have been widely adopted in HTC through a fine-tuning paradigm. However, in this paradigm, there exists a huge gap between the classification tasks with sophisticated label hierarchy and the masked language model (MLM) pretraining tasks of PLMs and thus the potentials of PLMs can not be fully tapped. To bridge the gap, in this paper, we propose HPT, a Hierarchy-aware Prompt Tuning method to handle HTC from a multi-label MLM perspective. Specifically, we construct a dynamic virtual template and label words that take the form of soft prompts to fuse the label hierarchy knowledge and introduce a zero-bounded multi-label cross entropy loss to harmonize the objectives of HTC and MLM. Extensive experiments show HPT achieves state-of-the-art performances on 3 popular HTC datasets and is adept at handling the imbalance and low resource situations. Our code is available at https://github.com/wzh9969/HPT.