Multi-objective Reinforcement Learning: A Tool for Pluralistic Alignment
This is an incremental overview paper discussing a domain-specific problem for AI alignment and stakeholders.
The paper addresses the challenge of aligning AI systems with multiple conflicting values or stakeholders by proposing multi-objective reinforcement learning (MORL) as an alternative to scalar RL, but does not provide specific results or numbers.
Reinforcement learning (RL) is a valuable tool for the creation of AI systems. However it may be problematic to adequately align RL based on scalar rewards if there are multiple conflicting values or stakeholders to be considered. Over the last decade multi-objective reinforcement learning (MORL) using vector rewards has emerged as an alternative to standard, scalar RL. This paper provides an overview of the role which MORL can play in creating pluralistically-aligned AI.