Keep it Simple: Data-efficient Learning for Controlling Complex Systems with Simple Models
This work provides a data-efficient learning method for controlling complex systems with high-dimensional state spaces from images, which is particularly beneficial for robotic manipulation of deformable objects where state representations are not readily available.
This paper addresses the challenge of controlling complex systems like deformable objects without explicit state representations, which typically requires extensive data. The proposed method, Learned Visual Similarity Predictive Control (LVSPC), achieves comparable performance to state-of-the-art reinforcement learning methods using an order of magnitude less data and successfully manipulates a rope on a real robot with an 80% success rate after only 10 trials.
When manipulating a novel object with complex dynamics, a state representation is not always available, for example for deformable objects. Learning both a representation and dynamics from observations requires large amounts of data. We propose Learned Visual Similarity Predictive Control (LVSPC), a novel method for data-efficient learning to control systems with complex dynamics and high-dimensional state spaces from images. LVSPC leverages a given simple model approximation from which image observations can be generated. We use these images to train a perception model that estimates the simple model state from observations of the complex system online. We then use data from the complex system to fit the parameters of the simple model and learn where this model is inaccurate, also online. Finally, we use Model Predictive Control and bias the controller away from regions where the simple model is inaccurate and thus where the controller is less reliable. We evaluate LVSPC on two tasks; manipulating a tethered mass and a rope. We find that our method performs comparably to state-of-the-art reinforcement learning methods with an order of magnitude less data. LVSPC also completes the rope manipulation task on a real robot with 80% success rate after only 10 trials, despite using a perception system trained only on images from simulation.