AI Meets Physical World -- Exploring Robot Cooking
This work addresses the challenge of bridging AI with physical robotics for complex tasks like cooking, representing an incremental advancement in robot manipulation.
The paper tackles the problem of enabling robots to perform physically interactive manipulation tasks by learning from online instructional videos, resulting in a system that can interpret high-level commands and generate executable motion sequences for tasks like cooking.
This paper describes our recent research effort to bring the computer intelligence into the physical world so that robots could perform physically interactive manipulation tasks. Our proposed approach first gives robots the ability to learn manipulation skills by "watching" online instructional videos. After "watching" over 200 instructional videos, a functional object-oriented network (FOON) is constructed to represent the observed manipulation skills. Using the network, robots can take a high-level task command such as "I want BBQ Ribs for dinner," decipher the task goal, seek the correct objects to operate on, and then generate and execute a sequence of manipulation motions that can deal with physical interactions in a new condition. To best facilitate manipulation motions in the physical world, we also developed new grasping strategies for robots to hold objects with a firm grasp to withstand the disturbance during physical interactions.