A Structured Literature Review on Traditional Approaches in Current Natural Language Processing
This work provides a structured review for NLP researchers and practitioners, highlighting the persistence of traditional techniques in the era of neural networks, but it is incremental as it synthesizes existing literature without new methods or data.
The paper surveyed recent publications in five NLP application scenarios to assess the continued use of traditional approaches, finding that all scenarios still incorporate such methods in various roles like pipelines or baselines.
The continued rise of neural networks and large language models in the more recent past has altered the natural language processing landscape, enabling new approaches towards typical language tasks and achieving mainstream success. Despite the huge success of large language models, many disadvantages still remain and through this work we assess the state of the art in five application scenarios with a particular focus on the future perspectives and sensible application scenarios of traditional and older approaches and techniques. In this paper we survey recent publications in the application scenarios classification, information and relation extraction, text simplification as well as text summarization. After defining our terminology, i.e., which features are characteristic for traditional techniques in our interpretation for the five scenarios, we survey if such traditional approaches are still being used, and if so, in what way they are used. It turns out that all five application scenarios still exhibit traditional models in one way or another, as part of a processing pipeline, as a comparison/baseline to the core model of the respective paper, or as the main model(s) of the paper. For the complete statistics, see https://zenodo.org/records/13683801