Benchmarking LLMs for Mimicking Child-Caregiver Language in Interaction
This work addresses the need for benchmarks in child-oriented AI applications, though it is incremental as it builds on existing LLM evaluation methods.
The paper tackled the problem of evaluating how well large language models (LLMs) can simulate early child-caregiver language interactions, finding that models like Llama 3 and GPT-4o approximate dialogues at word and utterance levels but struggle with discursive patterns, alignment, and diversity compared to humans.
LLMs can generate human-like dialogues, yet their ability to simulate early child-adult interactions remains largely unexplored. In this paper, we examined how effectively LLMs can capture the distinctive features of child-caregiver language in interaction, using both static and interactive benchmarking methods. We found that state-of-the-art LLMs like Llama 3 and GPT-4o can approximate child-caregiver dialogues at the word and utterance level, but they struggle to reproduce the child and caregiver's discursive patterns, exaggerate alignment, and fail to reach the level of diversity shown by humans. The broader goal of this work is to initiate the development of a comprehensive benchmark for LLMs in child-oriented applications.