Non-autoregressive Text Editing with Copy-aware Latent Alignments
This addresses the problem of slow and rigid text editing for NLP applications, offering a faster and more flexible solution, though it is incremental as it builds on existing Seq2Edit paradigms.
The authors tackled the slow inference and inflexibility of Seq2Seq and Seq2Edit models in text editing by proposing a non-autoregressive method using copy-aware latent CTC alignments, achieving similar or better results than Seq2Seq with over 4x speedup on GEC and sentence fusion tasks.
Recent work has witnessed a paradigm shift from Seq2Seq to Seq2Edit in the field of text editing, with the aim of addressing the slow autoregressive inference problem posed by the former. Despite promising results, Seq2Edit approaches still face several challenges such as inflexibility in generation and difficulty in generalizing to other languages. In this work, we propose a novel non-autoregressive text editing method to circumvent the above issues, by modeling the edit process with latent CTC alignments. We make a crucial extension to CTC by introducing the copy operation into the edit space, thus enabling more efficient management of textual overlap in editing. We conduct extensive experiments on GEC and sentence fusion tasks, showing that our proposed method significantly outperforms existing Seq2Edit models and achieves similar or even better results than Seq2Seq with over $4\times$ speedup. Moreover, it demonstrates good generalizability on German and Russian. In-depth analyses reveal the strengths of our method in terms of the robustness under various scenarios and generating fluent and flexible outputs.