Zachary Levonian

Zachary Levonian, Chenglu Li, Wangda Zhu et al.

For middle-school math students, interactive question-answering (QA) with tutors is an effective way to learn. The flexibility and emergent capabilities of generative large language models (LLMs) has led to a surge of interest in automating portions of the tutoring process - including interactive QA to support conceptual discussion of mathematical concepts. However, LLM responses to math questions can be incorrect or mismatched to the educational context - such as being misaligned with a school's curriculum. One potential solution is retrieval-augmented generation (RAG), which involves incorporating a vetted external knowledge source in the LLM prompt to increase response quality. In this paper, we designed prompts that retrieve and use content from a high-quality open-source math textbook to generate responses to real student questions. We evaluate the efficacy of this RAG system for middle-school algebra and geometry QA by administering a multi-condition survey, finding that humans prefer responses generated using RAG, but not when responses are too grounded in the textbook content. We argue that while RAG is able to improve response quality, designers of math QA systems must consider trade-offs between generating responses preferred by students and responses closely matched to specific educational resources.

Zachary Levonian, Marco Dow, Drew Erikson et al.

Online health communities offer the promise of support benefits to users, in particular because these communities enable users to find peers with similar experiences. Building mutually supportive connections between peers is a key motivation for using online health communities. However, a user's role in a community may influence the formation of peer connections. In this work, we study patterns of peer connections between two structural health roles: patient and non-professional caregiver. We examine user behavior in an online health community where finding peers is not explicitly supported. This context lets us use social network analysis methods to explore the growth of such connections in the wild and identify users' peer communication preferences. We investigated how connections between peers were initiated, finding that initiations are more likely between two authors who have the same role and who are close within the broader communication network. Relationships are also more likely to form and be more interactive when authors have the same role. Our results have implications for the design of systems supporting peer communication, e.g. peer-to-peer recommendation systems.

C. Estelle Smith, Zachary Levonian, Haiwei Ma et al.

Online Health Communities (OHCs) are known to provide substantial emotional and informational support to patients and family caregivers facing life-threatening diagnoses like cancer and other illnesses, injuries, or chronic conditions. Yet little work explores how OHCs facilitate other vital forms of social support, especially instrumental support. We partner with CaringBridge.org---a prominent OHC for journaling about health crises---to complete a two-phase study focused on instrumental support. Phase one involves a content analysis of 641 CaringBridge updates. Phase two is a survey of 991 CaringBridge users. Results show that patients and family caregivers diverge from their support networks in their preferences for specific instrumental support types. Furthermore, ``prayer support'' emerged as the most prominent support category across both phases. We discuss design implications to accommodate divergent preferences and to expand the instrumental support network. We also discuss the need for future work to empower family caregivers and to support spirituality.