Trust the AI, Doubt Yourself: The Effect of Urgency on Self-Confidence in Human-AI Interaction
This addresses a problem for software engineers and decision-makers by highlighting an incremental risk in AI system design that could undermine human performance and system sustainability.
The study investigated how urgency in human-AI interactions affects self-confidence, finding that while it doesn't impact trust in AI, it reduces users' self-confidence and self-efficacy, potentially leading to performance issues and unsustainable systems, based on an experiment with 30 participants.
Studies show that interactions with an AI system fosters trust in human users towards AI. An often overlooked element of such interaction dynamics is the (sense of) urgency when the human user is prompted by an AI agent, e.g., for advice or guidance. In this paper, we show that although the presence of urgency in human-AI interactions does not affect the trust in AI, it may be detrimental to the human user's self-confidence and self-efficacy. In the long run, the loss of confidence may lead to performance loss, suboptimal decisions, human errors, and ultimately, unsustainable AI systems. Our evidence comes from an experiment with 30 human participants. Our results indicate that users may feel more confident in their work when they are eased into the human-AI setup rather than exposed to it without preparation. We elaborate on the implications of this finding for software engineers and decision-makers.