Innate-Values-driven Reinforcement Learning based Cognitive Modeling
This addresses the challenge of developing AI agents that can self-organize and integrate into communities with safety and harmony, though it appears incremental as it builds on existing RL methods with added innate value components.
The paper tackles the problem of agents lacking intrinsic motivations in reinforcement learning by proposing innate-values-driven RL (IVRL), which incorporates innate values to balance internal and external utilities, and demonstrates that IVRL-based models achieve better performance than benchmark algorithms in the VIZDoom test platform.
Innate values describe agents' intrinsic motivations, which reflect their inherent interests and preferences for pursuing goals and drive them to develop diverse skills that satisfy their various needs. Traditional reinforcement learning (RL) is learning from interaction based on the feedback rewards of the environment. However, in real scenarios, the rewards are generated by agents' innate value systems, which differ vastly from individuals based on their needs and requirements. In other words, considering the AI agent as a self-organizing system, developing its awareness through balancing internal and external utilities based on its needs in different tasks is a crucial problem for individuals learning to support others and integrate community with safety and harmony in the long term. To address this gap, we propose a new RL model termed innate-values-driven RL (IVRL) based on combined motivations' models and expected utility theory to mimic its complex behaviors in the evolution through decision-making and learning. Then, we introduce two IVRL-based models: IV-DQN and IV-A2C. By comparing them with benchmark algorithms such as DQN, DDQN, A2C, and PPO in the Role-Playing Game (RPG) reinforcement learning test platform VIZDoom, we demonstrated that the IVRL-based models can help the agent rationally organize various needs, achieve better performance effectively.