"AI enhances our performance, I have no doubt this one will do the same": The Placebo effect is robust to negative descriptions of AI
This research addresses the problem of placebo effects in AI evaluation for users and developers, highlighting that expectations are biased and resistant to negative descriptions, which is incremental as it builds on prior work on placebo effects in human-AI interactions.
The study investigated whether negative descriptions of AI could lower user expectations and performance in a human-AI interaction task, finding that participants' high expectations and improved performance persisted regardless of the AI description, with no AI actually present. Concrete results included a Bayesian analysis showing robust expectations and cognitive modeling linking the advantage to increased information gathering.
Heightened AI expectations facilitate performance in human-AI interactions through placebo effects. While lowering expectations to control for placebo effects is advisable, overly negative expectations could induce nocebo effects. In a letter discrimination task, we informed participants that an AI would either increase or decrease their performance by adapting the interface, but in reality, no AI was present in any condition. A Bayesian analysis showed that participants had high expectations and performed descriptively better irrespective of the AI description when a sham-AI was present. Using cognitive modeling, we could trace this advantage back to participants gathering more information. A replication study verified that negative AI descriptions do not alter expectations, suggesting that performance expectations with AI are biased and robust to negative verbal descriptions. We discuss the impact of user expectations on AI interactions and evaluation and provide a behavioral placebo marker for human-AI interaction