Zombies in the Loop? Humans Trust Untrustworthy AI-Advisors for Ethical Decisions
This highlights a critical problem for AI ethics and safety, as users may rely on untrustworthy AI for sensitive decisions, suggesting incremental insights into human-AI interaction.
The study found that people trust AI advice on ethical decisions even when they lack information about its training data or receive information that should warrant distrust, indicating overtrust rather than distrust.
Departing from the claim that AI needs to be trustworthy, we find that ethical advice from an AI-powered algorithm is trusted even when its users know nothing about its training data and when they learn information about it that warrants distrust. We conducted online experiments where the subjects took the role of decision-makers who received advice from an algorithm on how to deal with an ethical dilemma. We manipulated the information about the algorithm and studied its influence. Our findings suggest that AI is overtrusted rather than distrusted. We suggest digital literacy as a potential remedy to ensure the responsible use of AI.