LC-Score: Reference-less estimation of Text Comprehension Difficulty
This addresses the lack of accurate metrics for text comprehension difficulty, which hinders accessibility efforts and Automatic Text Simplification model development, though it is incremental as it builds on existing readability and language guidelines.
The paper tackles the problem of estimating text comprehension difficulty without reference texts, introducing LC-Score to predict how easy a French text is to understand on a 0-100 scale, and finds that both indicator-based and neural approaches outperform existing metrics like FKGL in human evaluations.
Being able to read and understand written text is critical in a digital era. However, studies shows that a large fraction of the population experiences comprehension issues. In this context, further initiatives in accessibility are required to improve the audience text comprehension. However, writers are hardly assisted nor encouraged to produce easy-to-understand content. Moreover, Automatic Text Simplification (ATS) model development suffers from the lack of metric to accurately estimate comprehension difficulty We present \textsc{LC-Score}, a simple approach for training text comprehension metric for any French text without reference \ie predicting how easy to understand a given text is on a $[0, 100]$ scale. Our objective with this scale is to quantitatively capture the extend to which a text suits to the \textit{Langage Clair} (LC, \textit{Clear Language}) guidelines, a French initiative closely related to English Plain Language. We explore two approaches: (i) using linguistically motivated indicators used to train statistical models, and (ii) neural learning directly from text leveraging pre-trained language models. We introduce a simple proxy task for comprehension difficulty training as a classification task. To evaluate our models, we run two distinct human annotation experiments, and find that both approaches (indicator based and neural) outperforms commonly used readability and comprehension metrics such as FKGL.