AI Must not be Fully Autonomous
This addresses the ethical and safety concerns of autonomous AI for researchers and policymakers, but is incremental as it builds on existing discussions.
The paper argues against fully autonomous AI by identifying three levels of autonomy and presenting 12 arguments with counterarguments and rebuttals, supported by 15 pieces of evidence on AI risks like misaligned values.
Autonomous Artificial Intelligence (AI) has many benefits. It also has many risks. In this work, we identify the 3 levels of autonomous AI. We are of the position that AI must not be fully autonomous because of the many risks, especially as artificial superintelligence (ASI) is speculated to be just decades away. Fully autonomous AI, which can develop its own objectives, is at level 3 and without responsible human oversight. However, responsible human oversight is crucial for mitigating the risks. To ague for our position, we discuss theories of autonomy, AI and agents. Then, we offer 12 distinct arguments and 6 counterarguments with rebuttals to the counterarguments. We also present 15 pieces of recent evidence of AI misaligned values and other risks in the appendix.