Prompt- and Trait Relation-aware Cross-prompt Essay Trait Scoring
This addresses a real-education problem for automated essay scoring systems by enabling detailed trait scoring for essays on unseen prompts, though it is incremental as it builds on existing AES methods.
The paper tackled the challenge of cross-prompt essay trait scoring in automated essay scoring, where essays are written for unseen prompts and require detailed trait scores, by proposing a prompt- and trait relation-aware model that achieved state-of-the-art results across all prompts and traits, with significant improvements in low-resource settings.
Automated essay scoring (AES) aims to score essays written for a given prompt, which defines the writing topic. Most existing AES systems assume to grade essays of the same prompt as used in training and assign only a holistic score. However, such settings conflict with real-education situations; pre-graded essays for a particular prompt are lacking, and detailed trait scores of sub-rubrics are required. Thus, predicting various trait scores of unseen-prompt essays (called cross-prompt essay trait scoring) is a remaining challenge of AES. In this paper, we propose a robust model: prompt- and trait relation-aware cross-prompt essay trait scorer. We encode prompt-aware essay representation by essay-prompt attention and utilizing the topic-coherence feature extracted by the topic-modeling mechanism without access to labeled data; therefore, our model considers the prompt adherence of an essay, even in a cross-prompt setting. To facilitate multi-trait scoring, we design trait-similarity loss that encapsulates the correlations of traits. Experiments prove the efficacy of our model, showing state-of-the-art results for all prompts and traits. Significant improvements in low-resource-prompt and inferior traits further indicate our model's strength.