Position: We Need Practical AI Alignment Methods to Mirror Human Reasoning
For AI researchers and developers, this paper highlights the importance of cognitive alignment for user trust and adoption, but it is a position paper with no empirical results beyond survey data.
This position paper argues that AI systems in high-stakes decision-making should be cognitively aligned with human reasoning to improve understandability and trustworthiness. Survey data show many users find cognitive alignment essential, and the paper outlines gaps and a research agenda to achieve it.
AI systems are increasingly employed as decision aids, decision delegates, or autonomous decision-makers. This position paper argues that in many settings, particularly high-stakes decision-making, we need accurate cognitively-aligned AI systems that reason similarly to their users, and faithfully communicate their reasoning. We review evidence that cognitive alignment improves understandability and trustworthiness, and provide new survey data showing that many users find cognitive alignment "essential" when an AI's rationale for a judgment or action is important to them. We outline the gaps between existing alignment methods and what is needed to achieve cognitive alignment, and present a research agenda to address these gaps. We argue that cognitive misalignment represents a likely impediment to AI adoption in many envisioned applications, and that addressing it is important for creating AI systems on which users are both willing and justified to rely.