Semi-Supervised Text Simplification with Back-Translation and Asymmetric Denoising Autoencoders
This work addresses the challenge of limited parallel data in text simplification, offering a solution that benefits applications requiring accessible language, though it is incremental in its method adaptation.
The paper tackles the problem of text simplification by leveraging unpaired corpora, proposing asymmetric denoising autoencoders to improve performance, with results showing that the unsupervised model outperforms previous systems and the semi-supervised version is competitive with state-of-the-art methods.
Text simplification (TS) rephrases long sentences into simplified variants while preserving inherent semantics. Traditional sequence-to-sequence models heavily rely on the quantity and quality of parallel sentences, which limits their applicability in different languages and domains. This work investigates how to leverage large amounts of unpaired corpora in TS task. We adopt the back-translation architecture in unsupervised machine translation (NMT), including denoising autoencoders for language modeling and automatic generation of parallel data by iterative back-translation. However, it is non-trivial to generate appropriate complex-simple pair if we directly treat the set of simple and complex corpora as two different languages, since the two types of sentences are quite similar and it is hard for the model to capture the characteristics in different types of sentences. To tackle this problem, we propose asymmetric denoising methods for sentences with separate complexity. When modeling simple and complex sentences with autoencoders, we introduce different types of noise into the training process. Such a method can significantly improve the simplification performance. Our model can be trained in both unsupervised and semi-supervised manner. Automatic and human evaluations show that our unsupervised model outperforms the previous systems, and with limited supervision, our model can perform competitively with multiple state-of-the-art simplification systems.