Local Neural Descriptor Fields: Locally Conditioned Object Representations for Manipulation
This addresses the challenge for robots in household environments to handle a wide range of unfamiliar objects without extensive training.
The paper tackles the problem of generalizing robot manipulation skills from limited demonstrations to novel objects from unseen shape categories, achieving effective transfer in both simulation and real-world settings.
A robot operating in a household environment will see a wide range of unique and unfamiliar objects. While a system could train on many of these, it is infeasible to predict all the objects a robot will see. In this paper, we present a method to generalize object manipulation skills acquired from a limited number of demonstrations, to novel objects from unseen shape categories. Our approach, Local Neural Descriptor Fields (L-NDF), utilizes neural descriptors defined on the local geometry of the object to effectively transfer manipulation demonstrations to novel objects at test time. In doing so, we leverage the local geometry shared between objects to produce a more general manipulation framework. We illustrate the efficacy of our approach in manipulating novel objects in novel poses -- both in simulation and in the real world.