Building Knowledge from Interactions: An LLM-Based Architecture for Adaptive Tutoring and Social Reasoning
This work addresses the challenge of integrating robotics into everyday social and task-oriented interactions, such as tutoring, by enhancing LLM capabilities with memory and reasoning, though it appears incremental in building on existing LLM and HRI methods.
The paper tackles the problem of enabling robots to conduct adaptive, socially engaging interactions in tutoring or training scenarios by developing a multimodal, cognitively inspired framework that enhances LLM-based decision-making. The result is a system validated through a user study and offline experiments, showing improved management of complex interactions, autonomous task guidance, and contextual memory building.
Integrating robotics into everyday scenarios like tutoring or physical training requires robots capable of adaptive, socially engaging, and goal-oriented interactions. While Large Language Models show promise in human-like communication, their standalone use is hindered by memory constraints and contextual incoherence. This work presents a multimodal, cognitively inspired framework that enhances LLM-based autonomous decision-making in social and task-oriented Human-Robot Interaction. Specifically, we develop an LLM-based agent for a robot trainer, balancing social conversation with task guidance and goal-driven motivation. To further enhance autonomy and personalization, we introduce a memory system for selecting, storing and retrieving experiences, facilitating generalized reasoning based on knowledge built across different interactions. A preliminary HRI user study and offline experiments with a synthetic dataset validate our approach, demonstrating the system's ability to manage complex interactions, autonomously drive training tasks, and build and retrieve contextual memories, advancing socially intelligent robotics.