Multidimensional Perceptron for Efficient and Explainable Long Text Classification
This addresses efficiency and trust issues in sensitive domains like healthcare and legal text mining, though it is incremental as it builds on existing segmentation and encoding frameworks.
The paper tackles the problem of high cost and complexity in long text classification by proposing SWIPE, a model that replaces attention/RNNs with a multidimensional perceptron, achieving superior classification accuracy and efficiency while providing interpretability.
Because of the inevitable cost and complexity of transformer and pre-trained models, efficiency concerns are raised for long text classification. Meanwhile, in the highly sensitive domains, e.g., healthcare and legal long-text mining, potential model distrust, yet underrated and underexplored, may hatch vital apprehension. Existing methods generally segment the long text, encode each piece with the pre-trained model, and use attention or RNNs to obtain long text representation for classification. In this work, we propose a simple but effective model, Segment-aWare multIdimensional PErceptron (SWIPE), to replace attention/RNNs in the above framework. Unlike prior efforts, SWIPE can effectively learn the label of the entire text with supervised training, while perceive the labels of the segments and estimate their contributions to the long-text labeling in an unsupervised manner. As a general classifier, SWIPE can endorse different encoders, and it outperforms SOTA models in terms of classification accuracy and model efficiency. It is noteworthy that SWIPE achieves superior interpretability to transparentize long text classification results.