Unifying Question Answering, Text Classification, and Regression via Span Extraction
This work addresses a methodological bottleneck for NLP researchers by simplifying model architectures across diverse tasks.
The paper tackles the problem of disparate output layers across question answering, text classification, and regression tasks by proposing a unified span-extraction approach, achieving superior or comparable performance on multiple benchmarks.
Even as pre-trained language encoders such as BERT are shared across many tasks, the output layers of question answering, text classification, and regression models are significantly different. Span decoders are frequently used for question answering, fixed-class, classification layers for text classification, and similarity-scoring layers for regression tasks, We show that this distinction is not necessary and that all three can be unified as span extraction. A unified, span-extraction approach leads to superior or comparable performance in supplementary supervised pre-trained, low-data, and multi-task learning experiments on several question answering, text classification, and regression benchmarks.