Comparing Psychometric and Behavioral Predictors of Compliance During Human-AI Interactions
This provides a practical improvement for adaptive AI systems by suggesting cheaper, more accurate behavioral models over surveys.
The study compared self-reported trust inventories with behavioral measures for predicting compliance with AI recommendations, finding that behavioral predictors were more effective across three datasets.
Optimization of human-AI teams hinges on the AI's ability to tailor its interaction to individual human teammates. A common hypothesis in adaptive AI research is that minor differences in people's predisposition to trust can significantly impact their likelihood of complying with recommendations from the AI. Predisposition to trust is often measured with self-report inventories that are administered before interactions. We benchmark a popular measure of this kind against behavioral predictors of compliance. We find that the inventory is a less effective predictor of compliance than the behavioral measures in datasets taken from three previous research projects. This suggests a general property that individual differences in initial behavior are more predictive than differences in self-reported trust attitudes. This result also shows a potential for easily accessible behavioral measures to provide an AI with more accurate models without the use of (often costly) survey instruments.