Adrian Haimovich

Hasibur Rahman, Kenji Numata, Evelyn T Lai et al.

Serious illness conversations (SICs) align care with patients' values, goals, and preferences, yet they rarely occur in emergency departments (EDs), where time constraints and emotional burden often leave clinicians making high-stakes decisions without documented insight into what matters most to patients. We present a case study of ED GOAL-AI, a voice-based conversational agent for brief, structured values discussions with older adults in the ED, evaluated with 55 patients for feasibility and acceptability. Most participants completed the conversation and reported the interaction as acceptable and feasible, with ratings of feeling heard and understood comparable to clinicians. However, we also observed critical failure modes, including boundary violations such as hallucinated diagnostic statements, highlighting ethical and emotional risks. This work points to early promise for AI-mediated SICs while underscoring the need for careful boundary setting and participatory design before broader deployment.

Menglin Zhao, Zhuorui Yong, Ruijia Guan et al.

Serious illness conversations (SICs), discussions between clinical care teams and patients with serious, life-limiting illnesses about their values, goals, and care preferences, are critical for patient-centered care. Without these conversations, patients often receive aggressive interventions that may not align with their goals. Clinical care teams face significant barriers when conducting serious illness conversations with older adult patients in Emergency Department (ED) settings, where most older adult patients lack documented treatment goals. To understand current practices and identify AI support opportunities, we conducted interviews with two domain experts and nine ED clinical care team members. Through thematic analysis, we characterized a four-phase serious illness conversation workflow (identification, preparation, conduction, documentation) and identified key needs and challenges at each stage. Clinical care teams struggle with fragmented EHR data access, time constraints, emotional preparation demands, and documentation burdens. While participants expressed interest in AI tools for information synthesis, conversational support, and automated documentation, they emphasized preserving human connection and clinical autonomy. We present design guidelines for AI tools supporting SIC workflows that fit within existing clinical practices. This work contributes empirical understanding of ED-based serious illness conversations and provides design considerations for AI in high-stakes clinical environments.