Complex Word Identification: Challenges in Data Annotation and System Performance
This addresses challenges in natural language processing for tasks like text simplification, but it is incremental as it builds on prior shared task work.
The paper investigates the problem of complex word identification (CWI) by analyzing system performance on the SemEval CWI dataset, finding that most systems performed poorly, partly due to issues in human annotation methods.
This paper revisits the problem of complex word identification (CWI) following up the SemEval CWI shared task. We use ensemble classifiers to investigate how well computational methods can discriminate between complex and non-complex words. Furthermore, we analyze the classification performance to understand what makes lexical complexity challenging. Our findings show that most systems performed poorly on the SemEval CWI dataset, and one of the reasons for that is the way in which human annotation was performed.