Smart But Not Moral? Moral Alignment In Human-AI Decision-Making
For researchers and practitioners in AI ethics and human-AI interaction, it introduces a new perspective on alignment that emphasizes moral values, though it remains conceptual without empirical validation.
The paper argues that moral alignment—perceived congruence between AI decision logic and stakeholders' moral intuitions—is a fundamental dimension of human-AI decision-making, beyond functional alignment. It proposes a framework based on Moral Foundations Theory for multi-stakeholder contexts.
In high-stakes AI-supported decisions, considerations are not purely technical but involve moral judgments about fairness, responsibility, and harm. While prior research has focused mainly on functional or behavioral alignment, this paper argues that moral alignment may be a more fundamental dimension of human-AI decision-making. Moral alignment is defined as the perceived congruence between the values embedded in an AI system's decision logic and the moral intuitions of stakeholders. Building on Moral Foundations Theory, the paper adopts a multi-stakeholder perspective and highlights why moral (mis)alignment matters for the meaningful integration of AI in sensitive contexts.