Markerless Visual Robot Programming by Demonstration
This addresses the challenge of intuitive robot programming for non-experts in domestic settings, but appears incremental as it builds on existing pose estimation and ontology methods.
The paper tackles the problem of enabling robots to learn tasks from human demonstrations without markers by analyzing spatial constraints from human pose and object data, and demonstrates the approach in a kitchen meal preparation task.
In this paper we present an approach for learning to imitate human behavior on a semantic level by markerless visual observation. We analyze a set of spatial constraints on human pose data extracted using convolutional pose machines and object informations extracted from 2D image sequences. A scene analysis, based on an ontology of objects and affordances, is combined with continuous human pose estimation and spatial object relations. Using a set of constraints we associate the observed human actions with a set of executable robot commands. We demonstrate our approach in a kitchen task, where the robot learns to prepare a meal.