Why not both? Complementing explanations with uncertainty, and the role of self-confidence in Human-AI collaboration
This work addresses the problem of improving human-AI collaboration in high-stakes domains like healthcare and criminal justice, but it is incremental as it builds on existing research by combining uncertainty and explanations.
The study investigated how uncertainty estimates and model explanations affect users' reliance, understanding, and trust in AI models, and assessed the impact of users' self-confidence on their behavior in human-AI collaboration.
AI and ML models have already found many applications in critical domains, such as healthcare and criminal justice. However, fully automating such high-stakes applications can raise ethical or fairness concerns. Instead, in such cases, humans should be assisted by automated systems so that the two parties reach a joint decision, stemming out of their interaction. In this work we conduct an empirical study to identify how uncertainty estimates and model explanations affect users' reliance, understanding, and trust towards a model, looking for potential benefits of bringing the two together. Moreover, we seek to assess how users' behaviour is affected by their own self-confidence in their abilities to perform a certain task, while we also discuss how the latter may distort the outcome of an analysis based on agreement and switching percentages.