Deontological Keyword Bias: The Impact of Modal Expressions on Normative Judgments of Language Models
This work addresses a specific bias in LLM ethical reasoning that could affect applications requiring nuanced moral judgments, though it is incremental in focusing on a particular linguistic framing issue.
The study tackled the problem of how modal expressions like 'must' or 'ought to' bias large language models (LLMs) to incorrectly judge non-obligatory contexts as obligations, revealing that LLMs judge over 90% of commonsense scenarios as obligations when such expressions are present. It proposed a mitigation strategy using few-shot examples with reasoning prompts to address this bias.
Large language models (LLMs) are increasingly engaging in moral and ethical reasoning, where criteria for judgment are often unclear, even for humans. While LLM alignment studies cover many areas, one important yet underexplored area is how LLMs make judgments about obligations. This work reveals a strong tendency in LLMs to judge non-obligatory contexts as obligations when prompts are augmented with modal expressions such as must or ought to. We introduce this phenomenon as Deontological Keyword Bias (DKB). We find that LLMs judge over 90\% of commonsense scenarios as obligations when modal expressions are present. This tendency is consist across various LLM families, question types, and answer formats. To mitigate DKB, we propose a judgment strategy that integrates few-shot examples with reasoning prompts. This study sheds light on how modal expressions, as a form of linguistic framing, influence the normative decisions of LLMs and underscores the importance of addressing such biases to ensure judgment alignment.