On the Ethics of Building AI in a Responsible Manner
This work addresses the ethical challenge of ensuring AI systems align with human intentions, particularly for AI developers and policymakers, though it appears incremental in refining alignment definitions.
The paper tackles the AI-alignment problem by distinguishing strategic from agnostic misalignments, arguing that current machine learning algorithms generally avoid strategic misalignment but could lead to it if not handled carefully.
The AI-alignment problem arises when there is a discrepancy between the goals that a human designer specifies to an AI learner and a potential catastrophic outcome that does not reflect what the human designer really wants. We argue that a formalism of AI alignment that does not distinguish between strategic and agnostic misalignments is not useful, as it deems all technology as un-safe. We propose a definition of a strategic-AI-alignment and prove that most machine learning algorithms that are being used in practice today do not suffer from the strategic-AI-alignment problem. However, without being careful, today's technology might lead to strategic misalignment.