Why Would You Suggest That? Human Trust in Language Model Responses
This addresses the problem of human-AI trust in creative decision-making, highlighting nuanced trust dynamics that are important for developers and users, though it is incremental in exploring explanation effects.
The study investigated how explanations in LLM responses affect human trust in a news headline generation task, finding that explanations increase trust when responses are compared but not when shown independently, with users equally trusting deceptive responses in isolation.
The emergence of Large Language Models (LLMs) has revealed a growing need for human-AI collaboration, especially in creative decision-making scenarios where trust and reliance are paramount. Through human studies and model evaluations on the open-ended News Headline Generation task from the LaMP benchmark, we analyze how the framing and presence of explanations affect user trust and model performance. Overall, we provide evidence that adding an explanation in the model response to justify its reasoning significantly increases self-reported user trust in the model when the user has the opportunity to compare various responses. Position and faithfulness of these explanations are also important factors. However, these gains disappear when users are shown responses independently, suggesting that humans trust all model responses, including deceptive ones, equitably when they are shown in isolation. Our findings urge future research to delve deeper into the nuanced evaluation of trust in human-machine teaming systems.