Evaluating Small Decoder-Only Language Models for Grammar Correction and Text Simplification
This work addresses the problem of high computational costs for large language models in grammar correction and text simplification, but it is incremental as it shows current small models are not yet competitive.
The paper investigated whether small decoder-only language models could serve as efficient alternatives for grammar correction and text simplification, finding that while they learn some behaviors, their performance remains below strong baselines and current large language models, with issues like meaning retention and hallucinations.
Large language models have become extremely popular recently due to their ability to achieve strong performance on a variety of tasks, such as text generation and rewriting, but their size and computation cost make them difficult to access, deploy, and secure in many settings. This paper investigates whether small, decoder-only language models can provide an efficient alternative for the tasks of grammar correction and text simplification. The experiments in this paper focus on testing small language models out of the box, fine-tuned, and run sequentially on the JFLEG and ASSET datasets using established metrics. The results show that while SLMs may learn certain behaviors well, their performance remains below strong baselines and current LLMs. The results also show that SLMs struggle with retaining meaning and hallucinations. These findings suggest that despite their efficiency advantages, current SLMs are not yet competitive enough with modern LLMs for rewriting, and further advances in training are required for SLMs to close the performance gap between them and today's LLMs.