Demanding and Designing Aligned Cognitive Architectures
It addresses the problem of AI alignment for humanity, but is incremental as it builds on existing debates and frameworks.
The paper tackles the challenge of aligning AI systems with human goals by examining stakeholder demands, proposing cognitive architectures that combine reward maximization with other elements, and addressing interactions with machine learning researchers to identify unfashionable but tractable alignment options.
With AI systems becoming more powerful and pervasive, there is increasing debate about keeping their actions aligned with the broader goals and needs of humanity. This multi-disciplinary and multi-stakeholder debate must resolve many issues, here we examine three of them. The first issue is to clarify what demands stakeholders might usefully make on the designers of AI systems, useful because the technology exists to implement them. We make this technical topic more accessible by using the framing of cognitive architectures. The second issue is to move beyond an analytical framing that treats useful intelligence as being reward maximization only. To support this move, we define several AI cognitive architectures that combine reward maximization with other technical elements designed to improve alignment. The third issue is how stakeholders should calibrate their interactions with modern machine learning researchers. We consider how current fashions in machine learning create a narrative pull that participants in technical and policy discussions should be aware of, so that they can compensate for it. We identify several technically tractable but currently unfashionable options for improving AI alignment.