Evaluation of mathematical questioning strategies using data collected through weak supervision
This work addresses the problem of data scarcity and labeling costs for teacher training in mathematics education, but it is incremental as it applies existing methods to a specific scenario.
The paper tackled the challenge of developing teacher questioning strategies by creating an AI-based classroom simulator to rehearse mathematical questioning skills, using a human-in-the-loop approach to collect a high-quality dataset and evaluate the system's usability.
A large body of research demonstrates how teachers' questioning strategies can improve student learning outcomes. However, developing new scenarios is challenging because of the lack of training data for a specific scenario and the costs associated with labeling. This paper presents a high-fidelity, AI-based classroom simulator to help teachers rehearse research-based mathematical questioning skills. Using a human-in-the-loop approach, we collected a high-quality training dataset for a mathematical questioning scenario. Using recent advances in uncertainty quantification, we evaluated our conversational agent for usability and analyzed the practicality of incorporating a human-in-the-loop approach for data collection and system evaluation for a mathematical questioning scenario.