Automatically assessing oral narratives of Afrikaans and isiXhosa children
This addresses the challenge for teachers in large preschool classrooms to accurately assess narrative skills, though it is incremental as it builds on existing speech recognition and machine learning methods.
The paper tackles the problem of identifying preschool children needing literacy intervention by developing an automatic system to assess oral narratives in Afrikaans and isiXhosa, using an LLM-based scoring model that performs comparably to a human expert in flagging children.
Developing narrative and comprehension skills in early childhood is critical for later literacy. However, teachers in large preschool classrooms struggle to accurately identify students who require intervention. We present a system for automatically assessing oral narratives of preschool children in Afrikaans and isiXhosa. The system uses automatic speech recognition followed by a machine learning scoring model to predict narrative and comprehension scores. For scoring predicted transcripts, we compare a linear model to a large language model (LLM). The LLM-based system outperforms the linear model in most cases, but the linear system is competitive despite its simplicity. The LLM-based system is comparable to a human expert in flagging children who require intervention. We lay the foundation for automatic oral assessments in classrooms, giving teachers extra capacity to focus on personalised support for children's learning.