Quantifying Divergence for Human-AI Collaboration and Cognitive Trust
This addresses the challenge of integrating human factors into AI trust and collaboration for users in human-AI interaction domains, but it is incremental as it builds on existing divergence metrics.
The paper tackled the problem of predicting human-AI collaboration likelihood and cognitive trust by proposing decision-making similarity measures based on divergence metrics, finding that people tend to collaborate with models most similar to them (measured via JSD) but this does not imply similar cognitive trust levels.
Predicting the collaboration likelihood and measuring cognitive trust to AI systems is more important than ever. To do that, previous research mostly focus solely on the model features (e.g., accuracy, confidence) and ignore the human factor. To address that, we propose several decision-making similarity measures based on divergence metrics (e.g., KL, JSD) calculated over the labels acquired from humans and a wide range of models. We conduct a user study on a textual entailment task, where the users are provided with soft labels from various models and asked to pick the closest option to them. The users are then shown the similarities/differences to their most similar model and are surveyed for their likelihood of collaboration and cognitive trust to the selected system. Finally, we qualitatively and quantitatively analyze the relation between the proposed decision-making similarity measures and the survey results. We find that people tend to collaborate with their most similar models -- measured via JSD -- yet this collaboration does not necessarily imply a similar level of cognitive trust. We release all resources related to the user study (e.g., design, outputs), models, and metrics at our repo.