Bounded Minds, Generative Machines: Envisioning Conversational AI that Works with Human Heuristics and Reduces Bias Risk
This work addresses the issue of bias and poor decision-making in conversational AI for users with bounded rationality, but it is incremental as it outlines a pathway rather than presenting new results.
The paper tackles the problem of conversational AI assuming idealized users by proposing a design approach that works with human heuristics to reduce bias risk, outlining research directions for detecting cognitive vulnerabilities and evaluating systems based on decision quality.
Conversational AI is rapidly becoming a primary interface for information seeking and decision making, yet most systems still assume idealized users. In practice, human reasoning is bounded by limited attention, uneven knowledge, and reliance on heuristics that are adaptive but bias-prone. This article outlines a research pathway grounded in bounded rationality, and argues that conversational AI should be designed to work with human heuristics rather than against them. It identifies key directions for detecting cognitive vulnerability, supporting judgment under uncertainty, and evaluating conversational systems beyond factual accuracy, toward decision quality and cognitive robustness.