Teaching Robots to Build Simulations of Themselves
This work addresses the challenge of robot autonomy and self-awareness by reducing reliance on extensive real-world data and kinematic priors, representing a novel advancement in robotic simulation.
The paper tackles the problem of enabling robots to autonomously model and predict their own dynamics using only raw video data, resulting in a self-supervised framework that achieves accurate motion planning and allows robots to detect abnormalities and recover from damage.
The emergence of vision catalysed a pivotal evolutionary advancement, enabling organisms not only to perceive but also to interact intelligently with their environment. This transformation is mirrored by the evolution of robotic systems, where the ability to leverage vision to simulate and predict their own dynamics marks a leap towards autonomy and self-awareness. Humans utilize vision to record experiences and internally simulate potential actions. For example, we can imagine that, if we stand up and raise our arms, the body will form a T shape without physical movement. Similarly, simulation allows robots to plan and predict the outcomes of potential actions without execution. Here we introduce a self-supervised learning framework to enable robots to model and predict their morphology, kinematics and motor control using only brief raw video data, eliminating the need for extensive real-world data collection and kinematic priors. By observing their own movements, akin to humans watching their reflection in a mirror, robots learn an ability to simulate themselves and predict their spatial motion for various tasks. Our results demonstrate that this self-learned simulation not only enables accurate motion planning but also allows the robot to detect abnormalities and recover from damage.