FRAC-Q-Learning: A Reinforcement Learning with Boredom Avoidance Processes for Social Robots
This addresses boredom avoidance for users of social robots, but it is incremental as it builds on existing Q-learning with specialized modifications.
The paper tackled the problem of user boredom in social robots by proposing FRAC-Q-learning, a reinforcement learning method with forgetting, randomizing, and categorizing processes, which showed significantly higher interest scores and harder-to-bore users compared to traditional Q-learning.
The reinforcement learning algorithms have often been applied to social robots. However, most reinforcement learning algorithms were not optimized for the use of social robots, and consequently they may bore users. We proposed a new reinforcement learning method specialized for the social robot, the FRAC-Q-learning, that can avoid user boredom. The proposed algorithm consists of a forgetting process in addition to randomizing and categorizing processes. This study evaluated interest and boredom hardness scores of the FRAC-Q-learning by a comparison with the traditional Q-learning. The FRAC-Q-learning showed significantly higher trend of interest score, and indicated significantly harder to bore users compared to the traditional Q-learning. Therefore, the FRAC-Q-learning can contribute to develop a social robot that will not bore users. The proposed algorithm has a potential to apply for Web-based communication and educational systems. This paper presents the entire process, detailed implementation and a detailed evaluation method of the of the FRAC-Q-learning for the first time.