WikiSQE: A Large-Scale Dataset for Sentence Quality Estimation in Wikipedia
This provides a resource for NLP tasks to assist in Wikipedia editing by addressing quality estimation, though it is incremental as it focuses on dataset creation rather than a new method.
The authors tackled the problem of identifying poor-quality sentences in Wikipedia by creating WikiSQE, a large-scale dataset with about 3.4 million sentences and 153 quality labels, and found that certain issues like citation or syntax were harder to detect automatically, with their model outperforming human annotators.
Wikipedia can be edited by anyone and thus contains various quality sentences. Therefore, Wikipedia includes some poor-quality edits, which are often marked up by other editors. While editors' reviews enhance the credibility of Wikipedia, it is hard to check all edited text. Assisting in this process is very important, but a large and comprehensive dataset for studying it does not currently exist. Here, we propose WikiSQE, the first large-scale dataset for sentence quality estimation in Wikipedia. Each sentence is extracted from the entire revision history of English Wikipedia, and the target quality labels were carefully investigated and selected. WikiSQE has about 3.4 M sentences with 153 quality labels. In the experiment with automatic classification using competitive machine learning models, sentences that had problems with citation, syntax/semantics, or propositions were found to be more difficult to detect. In addition, by performing human annotation, we found that the model we developed performed better than the crowdsourced workers. WikiSQE is expected to be a valuable resource for other tasks in NLP.