Prototyping three key properties of specific curiosity in computational reinforcement learning
This work addresses the challenge of integrating human-like curiosity into machine learning agents, which could enhance decision-making in complex environments, but it is incremental as it builds on existing curiosity research.
The authors tackled the problem of implementing specific curiosity in reinforcement learning agents by prototyping three key properties—directedness, cessation when satisfied, and voluntary exposure—in a proof-of-concept agent, demonstrating its adaptive behavior in a simple grid-world environment.
Curiosity for machine agents has been a focus of intense research. The study of human and animal curiosity, particularly specific curiosity, has unearthed several properties that would offer important benefits for machine learners, but that have not yet been well-explored in machine intelligence. In this work, we introduce three of the most immediate of these properties -- directedness, cessation when satisfied, and voluntary exposure -- and show how they may be implemented together in a proof-of-concept reinforcement learning agent; further, we demonstrate how the properties manifest in the behaviour of this agent in a simple non-episodic grid-world environment that includes curiosity-inducing locations and induced targets of curiosity. As we would hope, the agent exhibits short-term directed behaviour while updating long-term preferences to adaptively seek out curiosity-inducing situations. This work therefore presents a novel view into how specific curiosity operates and in the future might be integrated into the behaviour of goal-seeking, decision-making agents in complex environments.