Empathic Prompting: Non-Verbal Context Integration for Multimodal LLM Conversations
This is an incremental improvement for chatbot-mediated communication in domains like healthcare or education, where emotional signals are important but often missing in verbal exchanges.
The paper tackled the problem of enriching multimodal LLM conversations by integrating non-verbal emotional cues from facial expressions, resulting in a system that consistently incorporated these inputs into coherent outputs and improved conversational fluidity in a small usability evaluation (N=5).
We present Empathic Prompting, a novel framework for multimodal human-AI interaction that enriches Large Language Model (LLM) conversations with implicit non-verbal context. The system integrates a commercial facial expression recognition service to capture users' emotional cues and embeds them as contextual signals during prompting. Unlike traditional multimodal interfaces, empathic prompting requires no explicit user control; instead, it unobtrusively augments textual input with affective information for conversational and smoothness alignment. The architecture is modular and scalable, allowing integration of additional non-verbal modules. We describe the system design, implemented through a locally deployed DeepSeek instance, and report a preliminary service and usability evaluation (N=5). Results show consistent integration of non-verbal input into coherent LLM outputs, with participants highlighting conversational fluidity. Beyond this proof of concept, empathic prompting points to applications in chatbot-mediated communication, particularly in domains like healthcare or education, where users' emotional signals are critical yet often opaque in verbal exchanges.