Real-time Active Vision for a Humanoid Soccer Robot Using Deep Reinforcement Learning
This work is significant for humanoid soccer robots, specifically in the RoboCup context, by providing a more robust active vision system that is less susceptible to self-localization errors.
This paper tackles the problem of active vision for humanoid soccer robots, where the robot needs to optimize its viewpoint to track the ball and acquire landmarks for self-localization. The authors propose a Deep Q-learning method that uses raw camera images to achieve an 80% success rate in finding the best viewpoint, outperforming entropy-based methods, especially in scenarios with high self-localization errors.
In this paper, we present an active vision method using a deep reinforcement learning approach for a humanoid soccer-playing robot. The proposed method adaptively optimises the viewpoint of the robot to acquire the most useful landmarks for self-localisation while keeping the ball into its viewpoint. Active vision is critical for humanoid decision-maker robots with a limited field of view. To deal with an active vision problem, several probabilistic entropy-based approaches have previously been proposed which are highly dependent on the accuracy of the self-localisation model. However, in this research, we formulate the problem as an episodic reinforcement learning problem and employ a Deep Q-learning method to solve it. The proposed network only requires the raw images of the camera to move the robot's head toward the best viewpoint. The model shows a very competitive rate of 80% success rate in achieving the best viewpoint. We implemented the proposed method on a humanoid robot simulated in Webots simulator. Our evaluations and experimental results show that the proposed method outperforms the entropy-based methods in the RoboCup context, in cases with high self-localisation errors.