Ethical Reasoning over Moral Alignment: A Case and Framework for In-Context Ethical Policies in LLMs
This addresses the challenge of ethical decision-making in AI for global applications, though it is incremental as it builds on existing LLM capabilities.
The paper tackles the problem of moral alignment in LLMs by proposing to infuse generic ethical reasoning capabilities to handle value pluralism globally, with initial experiments showing GPT-4 performs nearly perfectly but exhibits biases towards Western and English-speaking societies.
In this position paper, we argue that instead of morally aligning LLMs to specific set of ethical principles, we should infuse generic ethical reasoning capabilities into them so that they can handle value pluralism at a global scale. When provided with an ethical policy, an LLM should be capable of making decisions that are ethically consistent to the policy. We develop a framework that integrates moral dilemmas with moral principles pertaining to different foramlisms of normative ethics, and at different levels of abstractions. Initial experiments with GPT-x models shows that while GPT-4 is a nearly perfect ethical reasoner, the models still have bias towards the moral values of Western and English speaking societies.