Ab Initio Particle-based Object Manipulation
This addresses the problem of enabling robots to manipulate diverse objects in real-time without extensive pre-training, though it is an incremental improvement combining existing approaches.
The paper introduces Particle-based Object Manipulation (Prompt), a method for robots to manipulate novel objects without prior models or training data, using a particle-based representation learned online from visual input. Experiments show Prompt successfully handles various everyday and transparent objects, outperforming a state-of-the-art data-driven grasping method without offline training.
This paper presents Particle-based Object Manipulation (Prompt), a new approach to robot manipulation of novel objects ab initio, without prior object models or pre-training on a large object data set. The key element of Prompt is a particle-based object representation, in which each particle represents a point in the object, the local geometric, physical, and other features of the point, and also its relation with other particles. Like the model-based analytic approaches to manipulation, the particle representation enables the robot to reason about the object's geometry and dynamics in order to choose suitable manipulation actions. Like the data-driven approaches, the particle representation is learned online in real-time from visual sensor input, specifically, multi-view RGB images. The particle representation thus connects visual perception with robot control. Prompt combines the benefits of both model-based reasoning and data-driven learning. We show empirically that Prompt successfully handles a variety of everyday objects, some of which are transparent. It handles various manipulation tasks, including grasping, pushing, etc,. Our experiments also show that Prompt outperforms a state-of-the-art data-driven grasping method on the daily objects, even though it does not use any offline training data.