Digital Comprehensibility Assessment of Simplified Texts among Persons with Intellectual Disabilities
This work addresses the gap in evaluating text simplification models for persons with intellectual disabilities, an incremental improvement in assessment methods.
The study evaluated text comprehensibility for persons with intellectual disabilities by comparing unsimplified, automatically simplified, and manually simplified German texts, finding that comprehension questions were the most reliable measure for this group while reading speed offered insights into reading behavior.
Text simplification refers to the process of increasing the comprehensibility of texts. Automatic text simplification models are most commonly evaluated by experts or crowdworkers instead of the primary target groups of simplified texts, such as persons with intellectual disabilities. We conducted an evaluation study of text comprehensibility including participants with and without intellectual disabilities reading unsimplified, automatically and manually simplified German texts on a tablet computer. We explored four different approaches to measuring comprehensibility: multiple-choice comprehension questions, perceived difficulty ratings, response time, and reading speed. The results revealed significant variations in these measurements, depending on the reader group and whether the text had undergone automatic or manual simplification. For the target group of persons with intellectual disabilities, comprehension questions emerged as the most reliable measure, while analyzing reading speed provided valuable insights into participants' reading behavior.