Content Knowledge Identification with Multi-Agent Large Language Models (LLMs)
This work addresses the need for scalable and interpretable teacher professional development tools, though it is incremental as it builds on existing multi-agent LLM methods for a specific domain.
The authors tackled the problem of automatically identifying teachers' mathematical content knowledge in asynchronous professional development systems, proposing a multi-agent LLM framework that achieved promising performance on the MaCKT dataset without human annotations.
Teachers' mathematical content knowledge (CK) is of vital importance and need in teacher professional development (PD) programs. Computer-aided asynchronous PD systems are the most recent proposed PD techniques, which aim to help teachers improve their PD equally with fewer concerns about costs and limitations of time or location. However, current automatic CK identification methods, which serve as one of the core techniques of asynchronous PD systems, face challenges such as diversity of user responses, scarcity of high-quality annotated data, and low interpretability of the predictions. To tackle these challenges, we propose a Multi-Agent LLMs-based framework, LLMAgent-CK, to assess the user responses' coverage of identified CK learning goals without human annotations. By taking advantage of multi-agent LLMs in strong generalization ability and human-like discussions, our proposed LLMAgent-CK presents promising CK identifying performance on a real-world mathematical CK dataset MaCKT. Moreover, our case studies further demonstrate the working of the multi-agent framework.