Toward Subtrait-Level Model Explainability in Automated Writing Evaluation
This work addresses the need for more detailed score explanations for educators and students in educational technology, but it is incremental as it builds on existing methods with limited performance gains.
The paper tackled the problem of improving transparency in automated writing evaluation by developing a prototype for subtrait-level explainability using generative language models, resulting in modest correlations between human and automated subtrait scores.
Subtrait (latent-trait components) assessment presents a promising path toward enhancing transparency of automated writing scores. We prototype explainability and subtrait scoring with generative language models and show modest correlation between human subtrait and trait scores, and between automated and human subtrait scores. Our approach provides details to demystify scores for educators and students.