Split and Rephrase: Better Evaluation and a Stronger Baseline
This addresses a specific NLP task for researchers, with incremental improvements in evaluation and modeling.
The paper tackled the problem of splitting and rephrasing complex sentences into shorter ones in NLP, showing that vanilla seq2seq models suffer from memorization due to data overlap, and introduced a new data split and copy-mechanism models that outperformed the best baseline by 8.68 BLEU.
Splitting and rephrasing a complex sentence into several shorter sentences that convey the same meaning is a challenging problem in NLP. We show that while vanilla seq2seq models can reach high scores on the proposed benchmark (Narayan et al., 2017), they suffer from memorization of the training set which contains more than 89% of the unique simple sentences from the validation and test sets. To aid this, we present a new train-development-test data split and neural models augmented with a copy-mechanism, outperforming the best reported baseline by 8.68 BLEU and fostering further progress on the task.