HiGen: Hierarchy-Aware Sequence Generation for Hierarchical Text Classification
This work addresses hierarchical text classification, a complex multi-label task with imbalanced data, by introducing a novel generation-based method, though it is incremental as it builds on existing language model approaches.
The paper tackled hierarchical text classification by proposing HiGen, a text-generation framework that uses dynamic document representations and a level-guided loss, achieving superior performance on datasets like ENZYME, WOS, and NYT, with specific gains in handling data imbalance.
Hierarchical text classification (HTC) is a complex subtask under multi-label text classification, characterized by a hierarchical label taxonomy and data imbalance. The best-performing models aim to learn a static representation by combining document and hierarchical label information. However, the relevance of document sections can vary based on the hierarchy level, necessitating a dynamic document representation. To address this, we propose HiGen, a text-generation-based framework utilizing language models to encode dynamic text representations. We introduce a level-guided loss function to capture the relationship between text and label name semantics. Our approach incorporates a task-specific pretraining strategy, adapting the language model to in-domain knowledge and significantly enhancing performance for classes with limited examples. Furthermore, we present a new and valuable dataset called ENZYME, designed for HTC, which comprises articles from PubMed with the goal of predicting Enzyme Commission (EC) numbers. Through extensive experiments on the ENZYME dataset and the widely recognized WOS and NYT datasets, our methodology demonstrates superior performance, surpassing existing approaches while efficiently handling data and mitigating class imbalance. The data and code will be released publicly.