Integrating Transformer and Paraphrase Rules for Sentence Simplification
This work addresses the problem of generating simpler sentences while preserving meaning for applications like accessibility or language learning, representing an incremental improvement over existing methods.
The paper tackles sentence simplification by integrating a paraphrase knowledge base with a transformer model, resulting in a system that outperforms state-of-the-art baselines and selects more accurate simplification rules.
Sentence simplification aims to reduce the complexity of a sentence while retaining its original meaning. Current models for sentence simplification adopted ideas from ma- chine translation studies and implicitly learned simplification mapping rules from normal- simple sentence pairs. In this paper, we explore a novel model based on a multi-layer and multi-head attention architecture and we pro- pose two innovative approaches to integrate the Simple PPDB (A Paraphrase Database for Simplification), an external paraphrase knowledge base for simplification that covers a wide range of real-world simplification rules. The experiments show that the integration provides two major benefits: (1) the integrated model outperforms multiple state- of-the-art baseline models for sentence simplification in the literature (2) through analysis of the rule utilization, the model seeks to select more accurate simplification rules. The code and models used in the paper are available at https://github.com/ Sanqiang/text_simplification.