Semantic Robot Programming for Goal-Directed Manipulation in Cluttered Scenes
This work addresses the challenge of intuitive robot programming for non-experts in cluttered environments, representing an incremental improvement by combining existing paradigms like programming by demonstration and semantic mapping.
The authors tackled the problem of programming robots for goal-directed manipulation in cluttered scenes by introducing Semantic Robot Programming (SRP), which allows users to demonstrate goal scenes for the robot to achieve, and demonstrated its efficacy in tray-setting tasks with a Fetch robot using a new perception method called DIGEST.
We present the Semantic Robot Programming (SRP) paradigm as a convergence of robot programming by demonstration and semantic mapping. In SRP, a user can directly program a robot manipulator by demonstrating a snapshot of their intended goal scene in workspace. The robot then parses this goal as a scene graph comprised of object poses and inter-object relations, assuming known object geometries. Task and motion planning is then used to realize the user's goal from an arbitrary initial scene configuration. Even when faced with different initial scene configurations, SRP enables the robot to seamlessly adapt to reach the user's demonstrated goal. For scene perception, we propose the Discriminatively-Informed Generative Estimation of Scenes and Transforms (DIGEST) method to infer the initial and goal states of the world from RGBD images. The efficacy of SRP with DIGEST perception is demonstrated for the task of tray-setting with a Michigan Progress Fetch robot. Scene perception and task execution are evaluated with a public household occlusion dataset and our cluttered scene dataset.