F-PABEE: Flexible-patience-based Early Exiting for Single-label and Multi-label text Classification Tasks
This work addresses efficiency bottlenecks in large language models for text classification tasks, offering an incremental improvement over prior early exiting methods.
The paper tackles computational complexity and overthinking in pre-trained language models by proposing F-PABEE, a flexible early exiting method for single-label and multi-label text classification, achieving better speedup-accuracy balance and faster inference with improved performance on models like BERT and ALBERT.
Computational complexity and overthinking problems have become the bottlenecks for pre-training language models (PLMs) with millions or even trillions of parameters. A Flexible-Patience-Based Early Exiting method (F-PABEE) has been proposed to alleviate the problems mentioned above for single-label classification (SLC) and multi-label classification (MLC) tasks. F-PABEE makes predictions at the classifier and will exit early if predicted distributions of cross-layer are consecutively similar. It is more flexible than the previous state-of-the-art (SOTA) early exiting method PABEE because it can simultaneously adjust the similarity score thresholds and the patience parameters. Extensive experiments show that: (1) F-PABEE makes a better speedup-accuracy balance than existing early exiting strategies on both SLC and MLC tasks. (2) F-PABEE achieves faster inference and better performances on different PLMs such as BERT and ALBERT. (3) F-PABEE-JSKD performs best for F-PABEE with different similarity measures.