Measuring Value Alignment
This addresses the critical issue of value alignment for AI developers and ethicists, though it appears incremental as it builds on existing MDP frameworks.
The paper tackles the problem of ensuring AI systems align with human values by introducing a formalism to quantify this alignment using Markov Decision Processes, enabling evaluation of preference changes across state transitions in a normative world.
As artificial intelligence (AI) systems become increasingly integrated into various domains, ensuring that they align with human values becomes critical. This paper introduces a novel formalism to quantify the alignment between AI systems and human values, using Markov Decision Processes (MDPs) as the foundational model. We delve into the concept of values as desirable goals tied to actions and norms as behavioral guidelines, aiming to shed light on how they can be used to guide AI decisions. This framework offers a mechanism to evaluate the degree of alignment between norms and values by assessing preference changes across state transitions in a normative world. By utilizing this formalism, AI developers and ethicists can better design and evaluate AI systems to ensure they operate in harmony with human values. The proposed methodology holds potential for a wide range of applications, from recommendation systems emphasizing well-being to autonomous vehicles prioritizing safety.