Evaluating LLMs on Generating Age-Appropriate Child-Like Conversations
This addresses the problem of LLMs generating authentic child-like dialogue for specialized applications, particularly in low-resource languages, but is incremental as it focuses on evaluation rather than new methods.
The study evaluated five LLMs for generating age-appropriate Norwegian child-like conversations, finding that most models produced linguistically advanced language for target ages 5 and 9, with GPT-4 and NorBloom-7b performing relatively well and evaluators showing higher accuracy for younger children.
Large Language Models (LLMs), predominantly trained on adult conversational data, face significant challenges when generating authentic, child-like dialogue for specialized applications. We present a comparative study evaluating five different LLMs (GPT-4, RUTER-LLAMA-2-13b, GPTSW, NorMistral-7b, and NorBloom-7b) to generate age-appropriate Norwegian conversations for children aged 5 and 9 years. Through a blind evaluation by eleven education professionals using both real child interview data and LLM-generated text samples, we assessed authenticity and developmental appropriateness. Our results show that evaluators achieved strong inter-rater reliability (ICC=0.75) and demonstrated higher accuracy in age prediction for younger children (5-year-olds) compared to older children (9-year-olds). While GPT-4 and NorBloom-7b performed relatively well, most models generated language perceived as more linguistically advanced than the target age groups. These findings highlight critical data-related challenges in developing LLM systems for specialized applications involving children, particularly in low-resource languages where comprehensive age-appropriate lexical resources are scarce.