Pretrained Generalized Autoregressive Model with Adaptive Probabilistic Label Clusters for Extreme Multi-label Text Classification
This addresses the problem of efficient and accurate tagging in large-scale label sets for text classification, representing an incremental improvement over existing methods.
The authors tackled extreme multi-label text classification by proposing APLC-XLNet, which fine-tunes XLNet and uses adaptive probabilistic label clusters to reduce computational time, achieving new state-of-the-art results on four out of five benchmark datasets.
Extreme multi-label text classification (XMTC) is a task for tagging a given text with the most relevant labels from an extremely large label set. We propose a novel deep learning method called APLC-XLNet. Our approach fine-tunes the recently released generalized autoregressive pretrained model (XLNet) to learn a dense representation for the input text. We propose Adaptive Probabilistic Label Clusters (APLC) to approximate the cross entropy loss by exploiting the unbalanced label distribution to form clusters that explicitly reduce the computational time. Our experiments, carried out on five benchmark datasets, show that our approach has achieved new state-of-the-art results on four benchmark datasets. Our source code is available publicly at https://github.com/huiyegit/APLC_XLNet.