ChID: A Large-scale Chinese IDiom Dataset for Cloze Test
This addresses the problem of limited corpora for Chinese language processing, specifically for cloze tests involving idioms, but it is incremental as it focuses on dataset creation rather than novel methods.
The authors tackled the lack of datasets for Chinese cloze-style reading comprehension by creating ChID, a large-scale dataset focused on idioms, and found that state-of-the-art models perform substantially worse than humans, indicating significant room for improvement.
Cloze-style reading comprehension in Chinese is still limited due to the lack of various corpora. In this paper we propose a large-scale Chinese cloze test dataset ChID, which studies the comprehension of idiom, a unique language phenomenon in Chinese. In this corpus, the idioms in a passage are replaced by blank symbols and the correct answer needs to be chosen from well-designed candidate idioms. We carefully study how the design of candidate idioms and the representation of idioms affect the performance of state-of-the-art models. Results show that the machine accuracy is substantially worse than that of human, indicating a large space for further research.