Robot Cooking with Stir-fry: Bimanual Non-prehensile Manipulation of Semi-fluid Objects
This work addresses the challenge of bimanual non-prehensile manipulation for deformable objects in robotics, with potential applications in cooking and other domains, though it is incremental in extending existing methods to a specific task.
The paper tackles the problem of teaching a bimanual robot to perform stir-fry cooking, a complex task involving dynamic manipulation of semi-fluid objects, by proposing a decoupled learning framework from human demonstrations and a Graph-Transformer model for coordination, achieving successful deployment on a real robot system.
This letter describes an approach to achieve well-known Chinese cooking art stir-fry on a bimanual robot system. Stir-fry requires a sequence of highly dynamic coordinated movements, which is usually difficult to learn for a chef, let alone transfer to robots. In this letter, we define a canonical stir-fry movement, and then propose a decoupled framework for learning this deformable object manipulation from human demonstration. First, the dual arms of the robot are decoupled into different roles (a leader and follower) and learned with classical and neural network-based methods separately, then the bimanual task is transformed into a coordination problem. To obtain general bimanual coordination, we secondly propose a Graph and Transformer based model -- Structured-Transformer, to capture the spatio-temporal relationship between dual-arm movements. Finally, by adding visual feedback of content deformation, our framework can adjust the movements automatically to achieve the desired stir-fry effect. We verify the framework by a simulator and deploy it on a real bimanual Panda robot system. The experimental results validate our framework can realize the bimanual robot stir-fry motion and have the potential to extend to other deformable objects with bimanual coordination.