Model-Free RL Agents Demonstrate System 1-Like Intentionality
This work addresses the problem of interpreting intentionality in AI systems for researchers and policymakers, though it is incremental in linking existing concepts from cognitive psychology to RL.
This paper argues that model-free reinforcement learning agents exhibit behaviors analogous to System 1 processes in human cognition, challenging the assumption that intentionality requires planning and suggesting it can arise from reactive behaviors, with implications for AI ethics and regulation.
This paper argues that model-free reinforcement learning (RL) agents, while lacking explicit planning mechanisms, exhibit behaviours that can be analogised to System 1 ("thinking fast") processes in human cognition. Unlike model-based RL agents, which operate akin to System 2 ("thinking slow") reasoning by leveraging internal representations for planning, model-free agents react to environmental stimuli without anticipatory modelling. We propose a novel framework linking the dichotomy of System 1 and System 2 to the distinction between model-free and model-based RL. This framing challenges the prevailing assumption that intentionality and purposeful behaviour require planning, suggesting instead that intentionality can manifest in the structured, reactive behaviours of model-free agents. By drawing on interdisciplinary insights from cognitive psychology, legal theory, and experimental jurisprudence, we explore the implications of this perspective for attributing responsibility and ensuring AI safety. These insights advocate for a broader, contextually informed interpretation of intentionality in RL systems, with implications for their ethical deployment and regulation.