Edith Law

Rachel Love, Edith Law, Philip R. Cohen et al.

Conversational teachable agents offer a promising platform to support learning, both in the classroom and in remote settings. In this context, the agent takes the role of the novice, while the student takes on the role of teacher. This framing is significant for its ability to elicit the Protégé effect in the student-teacher, a pedagogical phenomenon known to increase engagement in the teaching task, and also improve cognitive outcomes. In prior work, teachable agents often take a passive role in the learning interaction, and there are few studies in which the agent and student engage in natural language dialogue during the teaching task. This work investigates the effect of teaching modality when interacting with a virtual agent, via the web-based teaching platform, the Curiosity Notebook. A method of teaching the agent by selecting sentences from source material is compared to a method paraphrasing the source material and typing text input to teach. A user study has been conducted to measure the effect teaching modality on the learning outcomes and engagement of the participants. The results indicate that teaching via paraphrasing and text input has a positive effect on learning outcomes for the material covered, and also on aspects of affective engagement. Furthermore, increased paraphrasing effort, as measured by the similarity between the source material and the material the teacher conveyed to the robot, improves learning outcomes for participants.

Aakanksha Khandwaha, Edith Law

Despite AI tools becoming more prevalent and applicable to a variety of workplaces, workers consistently report uncertainty about where AI applies, what problems it can help solve, and how it fits into real workflows. In other words, there is a gap between `knowing' and `doing' when it comes to AI literacy. We propose an experiential form of AI literacy which integrates participant's daily experiences into the learning experience by brainstorming grounded AI use cases through storytelling. We introduce a novel pedagogical approach that helps individuals move away from abstract notions of AI towards practical knowledge of how AI would (or would not) work in different workflows, contexts, and situations. Through this approach, we anticipate two major outcomes: (1) enhanced AI literacy for stakeholders within a variety of work sectors and (2) concrete AI use cases developed through participatory design that are grounded in AI literacy and participant's expertise.

Anthony Maocheia-Ricci, Edith Law

Value-based approaches such as Value Sensitive Design (VSD) enable technology designers to engage with and integrate human values in technology through a tripartite methodology of conceptual, empirical, and technical investigations. However, VSD contains pitfalls in both translating values to requirements and a lack of normative grounding, leading to adaptations such as Jacobs' Capability Sensitive Design (CSD). Inspired by CSD and extensions of the design approach, we propose the concept of creating -Sensitive Design (-SD); a meta-framework to embed various political or ideological values as norms in a design research process. We exemplify this through \emph{Dependency}-Sensitive Design (DSD), combining ideas from Kittay's critiques of classical liberal theory within a practical VSD framework. Finally, we push for further work combining philosophy and design in areas beyond CSD and DSD.

Sheza Munir, Benjamin Mah, Krisha Kalsi et al.

In machine learning, "ground truth" refers to the assumed correct labels used to train and evaluate models. However, the foundational "ground truth" paradigm rests on a positivistic fallacy that treats human disagreement as technical noise rather than a vital sociotechnical signal. This systematic literature review analyzes research published between 2020 and 2025 across seven premier venues: ACL, AIES, CHI, CSCW, EAAMO, FAccT, and NeurIPS, investigating the mechanisms in data annotation practices that facilitate this "consensus trap". Our identification phase captured 30,897 records, which were refined via a tiered keyword filtration schema to a high-recall corpus of 3,042 records for manual screening, resulting in a final included corpus of 346 papers for qualitative synthesis. Our reflexive thematic analysis reveals that systemic failures in positional legibility, combined with the recent architectural shift toward human-as-verifier models, specifically the reliance on model-mediated annotations, introduce deep-seated anchoring bias and effectively remove human voices from the loop. We further demonstrate how geographic hegemony imposes Western norms as universal benchmarks, often enforced by the performative alignment of precarious data workers who prioritize requester compliance over honest subjectivity to avoid economic penalties. Critiquing the "noisy sensor" fallacy, where statistical models misdiagnose cultural pluralism as random error, we argue for reclaiming disagreement as a high-fidelity signal essential for building culturally competent models. To address these systemic tensions, we propose a roadmap for pluralistic annotation infrastructures that shift the objective from discovering a singular "right" answer to mapping the diversity of human experience.

Carter Blair, Kate Larson, Edith Law · harvard

AI agents are commonly aligned with "human values" through reinforcement learning from human feedback (RLHF), where a single reward model is learned from aggregated human feedback and used to align an agent's behavior. However, human values are not homogeneous--different people hold distinct and sometimes conflicting values. Aggregating feedback into a single reward model risks disproportionately suppressing minority preferences. To address this, we present a novel reward modeling approach for learning individualized reward models. Our approach uses a language model to guide users through reflective dialogues where they critique agent behavior and construct their preferences. This personalized dialogue history, containing the user's reflections and critiqued examples, is then used as context for another language model that serves as an individualized reward function (what we call a "verbal reward model") for evaluating new trajectories. In studies with 30 participants, our method achieved a 9-12% improvement in accuracy over non-reflective verbal reward models while being more sample efficient than traditional supervised learning methods.

Carter Blair, Kate Larson, Edith Law · harvard

Aligning AI agents with human values is challenging due to diverse and subjective notions of values. Standard alignment methods often aggregate crowd feedback, which can result in the suppression of unique or minority preferences. We introduce Interactive-Reflective Dialogue Alignment, a method that iteratively engages users in reflecting on and specifying their subjective value definitions. This system learns individual value definitions through language-model-based preference elicitation and constructs personalized reward models that can be used to align AI behaviour. We evaluated our system through two studies with 30 participants, one focusing on "respect" and the other on ethical decision-making in autonomous vehicles. Our findings demonstrate diverse definitions of value-aligned behaviour and show that our system can accurately capture each person's unique understanding. This approach enables personalized alignment and can inform more representative and interpretable collective alignment strategies.

Jessy Ceha, Edith Law, Dana Kulić et al.

With advances in artificial intelligence, research is increasingly exploring the potential functions that social robots can play in education. As teachers are a critical stakeholder in the use and application of educational technologies, we conducted a study to understand teachers' perspectives on how a social robot could support a variety of learning activities in the classroom. Through interviews, robot puppeteering, and group brainstorming sessions with five elementary and middle school teachers from a local school in Canada, we take a socio-technical perspective to conceptualize possible robot functions and behaviours, and the effects they may have on the current way learning activities are designed, planned, and executed. Overall, the teachers responded positively to the idea of introducing a social robot as a technological tool for learning activities, envisioning differences in usage for teacher-robot and student-robot interactions. Further, Engeström's Activity System Model -- a framework for analyzing human needs, tasks, and outcomes -- illustrated a number of tensions associated with learning activities in the classroom. We discuss the fine-grained robot functions and behaviours conceived by teachers, and how they address the current tensions -- providing suggestions for improving the design of social robots for learning activities.

Ken Jen Lee, Edith Law

In this position paper, we aim to spark more discussions surrounding the use of empathy as the intended outcome of many studies on immersive virtual reality experiences. As a construct, empathy has many significant flaws that may lead to unintended and negative outcomes, going against our original goal of employing these technologies for the betterment of society. We highlight the possible advantages of designing for rational compassion instead, and propose alternative research directions and outcome measurements for immersive virtual reality that urgently warrant our attention.

Jessy Ceha, Ken Jen Lee, Elizabeth Nilsen et al.

Previous studies have highlighted the benefits of pedagogical conversational agents using socially-oriented conversation with students. In this work, we examine the effects of a conversational agent's use of affiliative and self-defeating humour -- considered conducive to social well-being and enhancing interpersonal relationships -- on learners' perception of the agent and attitudes towards the task. Using a between-subjects protocol, 58 participants taught a conversational agent about rock classification using a learning-by-teaching platform, the Curiosity Notebook. While all agents were curious and enthusiastic, the style of humour was manipulated such that the agent either expressed an affiliative style, a self-defeating style, or no humour. Results demonstrate that affiliative humour can significantly increase motivation and effort, while self-defeating humour, although enhancing effort, negatively impacts enjoyment. Findings further highlight the importance of understanding learner characteristics when using humour.

Ken Jen Lee, Apoorva Chauhan, Joslin Goh et al.

While learning by teaching is a popular pedagogical technique, it is a learning phenomenon that is difficult to study due to variability in the tutor-tutee pairings and learning environments. In this paper, we introduce the Curiosity Notebook, a web-based research infrastructure for studying learning by teaching via the use of a teachable agent. We describe and provide rationale for the set of features that are essential for such a research infrastructure, outline how these features have evolved over two design iterations of the Curiosity Notebook and through two studies -- a 4-week field study with 12 elementary school students interacting with a NAO robot and an hour-long online observational study with 41 university students interacting with an agent -- demonstrate the utility of our platform for making observations of learning-by-teaching phenomena in diverse learning environments. Based on these findings, we conclude the paper by reflecting on our design evolution and envisioning future iterations of the Curiosity Notebook.

Nalin Chhibber, Edith Law

The traditional process of building interactive machine learning systems can be viewed as a teacher-learner interaction scenario where the machine-learners are trained by one or more human-teachers. In this work, we explore the idea of using a conversational interface to investigate the interaction between human-teachers and interactive machine-learners. Specifically, we examine whether teachable AI agents can reliably learn from human-teachers through conversational interactions, and how this learning compare with traditional supervised learning algorithms. Results validate the concept of teachable conversational agents and highlight the factors relevant for the development of machine learning systems that intend to learn from conversational interactions.

Henry Chen, Robin Cohen, Kerstin Dautenhahn et al.

Autonomous Vehicles (AV) will transform transportation, but also the interaction between vehicles and pedestrians. In the absence of a driver, it is not clear how an AV can communicate its intention to pedestrians. One option is to use visual signals. To advance their design, we conduct four human-participant experiments and evaluate six representative AV visual signals for visibility, intuitiveness, persuasiveness, and usability at pedestrian crossings. Based on the results, we distill twelve practical design recommendations for AV visual signals, with focus on signal pattern design and placement. Moreover, the paper advances the methodology for experimental evaluation of visual signals, including lab, closed-course, and public road tests using an autonomous vehicle. In addition, the paper also reports insights on pedestrian crosswalk behaviours and the impacts of pedestrian trust towards AVs on the behaviors. We hope that this work will constitute valuable input to the ongoing development of international standards for AV lamps, and thus help mature automated driving in general.

Glaucia Melo, Edith Law, Paulo Alencar et al.

Software development is a complex endeavor that depends on a wide variety of contextual factors involving a large amount of distributed information. This knowledge could include: technology-related tasks, software operating environments and stakeholder requirements. A major roadblock to using this knowledge in software development is that most of this information is implicit and captured in the developers' minds (tacit) or spread through volumes of documentation. Developers, as they work often have to maintain mental models of these tasks as they produce the software. As a result, context can be easily lost or forgotten and developers often use trial-and-error approaches while finishing the project. This study aims at analyzing whether supporting software developers with a chatbot during task execution can improve the overall development experience. The chatbot can assist the developers in executing different tasks based on implicit contextual information. We propose an implementation to explore the viability of using textual chatbots to assist developers automatically and proactively with software development project activities that recur.

Mehdi Alaimi, Edith Law, Kevin Daniel Pantasdo et al.

Question asking is an important tool for constructing academic knowledge, and a self-reinforcing driver of curiosity. However, research has found that question asking is infrequent in the classroom and children's questions are often superficial, lacking deep reasoning. In this work, we developed a pedagogical agent that encourages children to ask divergent-thinking questions, a more complex form of questions that is associated with curiosity. We conducted a study with 95 fifth grade students, who interacted with an agent that encourages either convergent-thinking or divergent-thinking questions. Results showed that both interventions increased the number of divergent-thinking questions and the fluency of question asking, while they did not significantly alter children's perception of curiosity despite their high intrinsic motivation scores. In addition, children's curiosity trait has a mediating effect on question asking under the divergent-thinking agent, suggesting that question-asking interventions must be personalized to each student based on their tendency to be curious.

Nalin Chhibber, Edith Law

Conversational agents are becoming increasingly popular for supporting and facilitating learning. Conventional pedagogical agents are designed to play the role of human teachers by giving instructions to the students. In this paper, we investigate the use of conversational agents to support the 'learning-by-teaching' paradigm where the agent receives instructions from students. In particular, we introduce Curiosity Notebook: an educational application that harvests conversational interventions to facilitate students' learning. Recognizing such interventions can not only help in engaging students within learning interactions, but also provide a deeper insight into the intricacies involved in designing conversational agents for educational purposes.

Nalin Chhibber, Rohail Syed, Mengqiu Teng et al.

Understanding how to engage users is a critical question in many applications. Previous research has shown that unexpected or astonishing events can attract user attention, leading to positive outcomes such as engagement and learning. In this work, we investigate the similarity and differences in how people and algorithms rank the surprisingness of facts. Our crowdsourcing study, involving 106 participants, shows that computational models of surprise can be used to artificially induce surprise in humans.