Why Sampling Is Not Choosing: Intentionality, Agency, and Moral Responsibility in Large Language Models
For philosophers and AI ethicists debating moral status of AI, this paper provides a clear argument against attributing moral responsibility to LLMs, though it is an incremental contribution to existing critiques.
The paper argues that large language models (LLMs) lack the intrinsic intentionality, commitment-bearing agency, and self-attributed action required for moral responsibility, despite generating coherent outputs. It concludes that attributions of agency or moral agency to LLMs are misguided.
Recent advances in large language models (LLMs) have prompted claims that such systems exhibit agency or qualify as moral agents. This paper argues that these attributions are misguided. We maintain that moral responsibility requires commitment-bearing agency grounded in intrinsic intentionality and self-attributed action, and that such agency constitutes the form of free will relevant to responsibility. Although LLMs generate coherent and normatively evaluable outputs, their operation is fully characterized by probabilistic input-output mappings learned from data. Their apparent intentionality is derived rather than intrinsic, and their outputs are neither owned as commitments nor guided by reasons. Variability introduced by stochastic sampling does not amount to choice or authorship. We address objections from the intentional stance, functionalism, compatibilism, and the presence of moral reasoning in model outputs, arguing that none suffice to establish genuine agency.