Self-supervised Learning of Image Embedding for Continuous Control
This work addresses the problem of task-specific and reward-dependent policies in robotics, offering a more generalizable approach, though it is incremental in building on existing reinforcement learning methods.
The paper tackles the challenge of robotic control from raw images by proposing a self-supervised method to learn general image embeddings and control primitives based on shortest-time state reachability, and introduces a new state-action value function structure that bridges model-free and model-based approaches, showing improved performance in three simulated robotic tasks.
Operating directly from raw high dimensional sensory inputs like images is still a challenge for robotic control. Recently, Reinforcement Learning methods have been proposed to solve specific tasks end-to-end, from pixels to torques. However, these approaches assume the access to a specified reward which may require specialized instrumentation of the environment. Furthermore, the obtained policy and representations tend to be task specific and may not transfer well. In this work we investigate completely self-supervised learning of a general image embedding and control primitives, based on finding the shortest time to reach any state. We also introduce a new structure for the state-action value function that builds a connection between model-free and model-based methods, and improves the performance of the learning algorithm. We experimentally demonstrate these findings in three simulated robotic tasks.