CLNov 25, 2022
GPT-3-driven pedagogical agents for training children's curious question-asking skillsRania Abdelghani, Yen-Hsiang Wang, Xingdi Yuan et al. · microsoft-research
In order to train children's ability to ask curiosity-driven questions, previous research has explored designing specific exercises relying on providing semantic and linguistic cues to help formulate such questions. But despite showing pedagogical efficiency, this method is still limited as it relies on generating the said cues by hand, which can be a very costly process. In this context, we propose to leverage advances in the natural language processing field (NLP) and investigate the efficiency of using a large language model (LLM) for automating the production of the pedagogical content of a curious question-asking (QA) training. We study generating the said content using the "prompt-based" method that consists of explaining the task to the LLM in natural text. We evaluate the output using human experts annotations and comparisons with hand-generated content. Results suggested indeed the relevance and usefulness of this content. We also conduct a field study in primary school (75 children aged 9-10), where we evaluate children's QA performance when having this training. We compare 3 types of content : 1) hand-generated content that proposes "closed" cues leading to predefined questions; 2) GPT-3-generated content that proposes the same type of cues; 3) GPT-3-generated content that proposes "open" cues leading to several possible questions. We see a similar QA performance between the two "closed" trainings (showing the scalability of the approach using GPT-3), and a better one for participants with the "open" training. These results suggest the efficiency of using LLMs to support children in generating more curious questions, using a natural language prompting approach that affords usability by teachers and other users not specialists of AI techniques. Furthermore, results also show that open-ended content may be more suitable for training curious question-asking skills.
CLSep 22, 2022
Selecting Better Samples from Pre-trained LLMs: A Case Study on Question GenerationXingdi Yuan, Tong Wang, Yen-Hsiang Wang et al. · microsoft-research
Large Language Models (LLMs) have in recent years demonstrated impressive prowess in natural language generation. A common practice to improve generation diversity is to sample multiple outputs from the model. However, there lacks a simple and robust way of selecting the best output from these stochastic samples. As a case study framed in the context of question generation, we propose two prompt-based approaches to selecting high-quality questions from a set of LLM-generated candidates. Our method works under the constraints of 1) a black-box (non-modifiable) question generation model and 2) lack of access to human-annotated references -- both of which are realistic limitations for real-world deployment of LLMs. With automatic as well as human evaluations, we empirically demonstrate that our approach can effectively select questions of higher qualities than greedy generation.
79.1CYApr 28
Curiosity and Metacognition: Towards a Unified Framework for Learning and Education in the Age of AIChloé Desvaux, Rania Abdelghani, Pierre-Yves Oudeyer et al.
This chapter examines the relationship between curiosity and metacognition as critical drivers of autonomous and self-regulated learning. We synthesize recent research to propose a unified framework integrating behavioral, computational, and psychoeducational dimensions, arguing that curiosity, i.e. the intrinsic drive to acquire new knowledge, relies fundamentally on metacognitive monitoring and control. From an educational perspective, we evaluate interventions designed to enhance curiosity in classroom settings. While promising, our review indicates that these interventions yield mixed results, often proving differentially effective for struggling learners, thereby underscoring the necessity for approaches tailored to individual profiles. Finally, we address the paradigm shift introduced by Generative AI. While Large Language Models (LLMs) offer unprecedented scalability for personalized inquiry, we argue that their default interaction modes pose significant risks to the dynamics of curiosity-driven learning. To mitigate these challenges, we review strategies to transform AI from a potential cognitive shortcut into a powerful partner for sustained epistemic development.
CYJan 16, 2024
Improved Performances and Motivation in Intelligent Tutoring Systems: Combining Machine Learning and Learner ChoiceBenjamin Clément, Hélène Sauzéon, Didier Roy et al.
Large class sizes challenge personalized learning in schools, prompting the use of educational technologies such as intelligent tutoring systems. To address this, we present an AI-driven personalization system, called ZPDES, based on the Learning Progress Hypothesis - modeling curiosity-driven learning - and multi-armed bandit techniques. It sequences exercises that maximize learning progress for each student. While previous studies demonstrated its efficacy in enhancing learning compared to hand-made curricula, its impact on student motivation remained unexplored. Furthermore, ZPDES previously lacked features allowing student choice, a limitation in agency that conflicts with its foundation on models of curiosity-driven learning. This study investigates how integrating choice, as a gamification element unrelated to exercise difficulty, affects both learning outcomes and motivation. We conducted an extensive field study (265 7-8 years old children, RCT design), comparing ZPDES with and without choice against a hand-designed curriculum. Results show that ZPDES improves both learning performance and the learning experience. Moreover adding choice to ZPDES enhances intrinsic motivation and further strengthens its learning benefits. In contrast, incorporating choice into a fixed, linear curriculum negatively impacts learning outcomes. These findings highlight that the intrinsic motivation elicited by choice (gamification) is beneficial only when paired with an adaptive personalized learning system. This insight is critical as gamified features become increasingly prevalent in educational technologies.
CVJan 26, 2022
Language-biased image classification: evaluation based on semantic representationsYoann Lemesle, Masataka Sawayama, Guillermo Valle-Perez et al.
Humans show language-biased image recognition for a word-embedded image, known as picture-word interference. Such interference depends on hierarchical semantic categories and reflects that human language processing highly interacts with visual processing. Similar to humans, recent artificial models jointly trained on texts and images, e.g., OpenAI CLIP, show language-biased image classification. Exploring whether the bias leads to interference similar to those observed in humans can contribute to understanding how much the model acquires hierarchical semantic representations from joint learning of language and vision. The present study introduces methodological tools from the cognitive science literature to assess the biases of artificial models. Specifically, we introduce a benchmark task to test whether words superimposed on images can distort the image classification across different category levels and, if it can, whether the perturbation is due to the shared semantic representation between language and vision. Our dataset is a set of word-embedded images and consists of a mixture of natural image datasets and hierarchical word labels with superordinate/basic category levels. Using this benchmark test, we evaluate the CLIP model. We show that presenting words distorts the image classification by the model across different category levels, but the effect does not depend on the semantic relationship between images and embedded words. This suggests that the semantic word representation in the CLIP visual processing is not shared with the image representation, although the word representation strongly dominates for word-embedded images.