MACAROON: Training Vision-Language Models To Be Your Engaged Partners
This addresses the issue of hallucinations and bias in LVLMs for users relying on accurate and safe AI interactions, representing a novel approach to enhancing model engagement.
The paper tackles the problem of large vision-language models (LVLMs) generating detailed responses to ambiguous or unanswerable questions, leading to hallucinations and bias, by shifting them from passive answer providers to proactive engaged partners that ask for clarifications. The result is MACAROON, a method that improves proactive engagement from an Aggregate Align Rate (AAR) of 0.28 in existing models to 0.84 while maintaining performance on general tasks.
Large vision-language models (LVLMs), while proficient in following instructions and responding to diverse questions, invariably generate detailed responses even when questions are ambiguous or unanswerable, leading to hallucinations and bias issues. Thus, it is essential for LVLMs to proactively engage with humans to ask for clarifications or additional information for better responses. In this study, we aim to shift LVLMs from passive answer providers to proactive engaged partners. We begin by establishing a three-tiered hierarchy for questions of invalid, ambiguous, and personalizable nature to measure the proactive engagement capabilities of LVLMs. Utilizing this hierarchy, we create PIE, (ProactIve Engagement Evaluation) through GPT-4o and human annotators, consisting of 853 questions across six distinct, fine-grained question types that are verified by human annotators and accompanied with well-defined metrics. Our evaluations on \benchmark indicate poor performance of existing LVLMs, with the best-performing open-weights model only achieving an Aggregate Align Rate (AAR) of 0.28. In response, we introduce MACAROON, self-iMaginAtion for ContrAstive pReference OptimizatiON, which instructs LVLMs to autonomously generate contrastive response pairs for unlabeled questions given the task description and human-crafted criteria. Then, the self-imagined data is formatted for conditional reinforcement learning. Experimental results show MACAROON effectively improves LVLMs' capabilities to be proactively engaged (0.84 AAR) while maintaining comparable performance on general tasks.