Purrfessor: A Fine-tuned Multimodal LLaVA Diet Health Chatbot
This is an incremental improvement for users seeking dietary advice through interactive AI chatbots.
This study tackled the problem of providing personalized dietary guidance by developing Purrfessor, a fine-tuned multimodal LLaVA chatbot, which significantly enhanced user perceptions of care (β=1.59, p=0.04) and interest (β=2.26, p=0.01) compared to a GPT-4 bot.
This study introduces Purrfessor, an innovative AI chatbot designed to provide personalized dietary guidance through interactive, multimodal engagement. Leveraging the Large Language-and-Vision Assistant (LLaVA) model fine-tuned with food and nutrition data and a human-in-the-loop approach, Purrfessor integrates visual meal analysis with contextual advice to enhance user experience and engagement. We conducted two studies to evaluate the chatbot's performance and user experience: (a) simulation assessments and human validation were conducted to examine the performance of the fine-tuned model; (b) a 2 (Profile: Bot vs. Pet) by 3 (Model: GPT-4 vs. LLaVA vs. Fine-tuned LLaVA) experiment revealed that Purrfessor significantly enhanced users' perceptions of care ($β= 1.59$, $p = 0.04$) and interest ($β= 2.26$, $p = 0.01$) compared to the GPT-4 bot. Additionally, user interviews highlighted the importance of interaction design details, emphasizing the need for responsiveness, personalization, and guidance to improve user engagement.