RewriteLM: An Instruction-Tuned Large Language Model for Text Rewriting
This work addresses the problem of text rewriting for users needing diverse wording and structures, though it is incremental as it builds on existing LLM methods with new tuning strategies.
The paper tackles the challenge of enabling large language models to perform cross-sentence text rewriting by developing instruction tuning and reinforcement learning strategies, resulting in significant improvements over various baselines as demonstrated on the new OpenRewriteEval benchmark.
Large Language Models (LLMs) have demonstrated impressive capabilities in creative tasks such as storytelling and E-mail generation. However, as LLMs are primarily trained on final text results rather than intermediate revisions, it might be challenging for them to perform text rewriting tasks. Most studies in the rewriting tasks focus on a particular transformation type within the boundaries of single sentences. In this work, we develop new strategies for instruction tuning and reinforcement learning to better align LLMs for cross-sentence rewriting tasks using diverse wording and structures expressed through natural languages including 1) generating rewriting instruction data from Wiki edits and public corpus through instruction generation and chain-of-thought prompting; 2) collecting comparison data for reward model training through a new ranking function. To facilitate this research, we introduce OpenRewriteEval, a novel benchmark covers a wide variety of rewriting types expressed through natural language instructions. Our results show significant improvements over a variety of baselines. The public repository is available on GitHub under Google Research (https://github.com/google-research/google-research/tree/master/rewritelm).