Geometry-aware RL for Manipulation of Varying Shapes and Deformable Objects
This work addresses a key problem in robotics for tasks like insertion and cloth manipulation, offering a novel approach with improved performance, though it appears incremental in building on existing graph and equivariant methods.
The paper tackles the challenge of manipulating objects with varying geometries and deformable objects in robotics by proposing a heterogeneous graph representation and a graph-based policy model called HEPi, which outperforms Transformer-based and non-heterogeneous equivariant policies in average returns, sample efficiency, and generalization to unseen objects.
Manipulating objects with varying geometries and deformable objects is a major challenge in robotics. Tasks such as insertion with different objects or cloth hanging require precise control and effective modelling of complex dynamics. In this work, we frame this problem through the lens of a heterogeneous graph that comprises smaller sub-graphs, such as actuators and objects, accompanied by different edge types describing their interactions. This graph representation serves as a unified structure for both rigid and deformable objects tasks, and can be extended further to tasks comprising multiple actuators. To evaluate this setup, we present a novel and challenging reinforcement learning benchmark, including rigid insertion of diverse objects, as well as rope and cloth manipulation with multiple end-effectors. These tasks present a large search space, as both the initial and target configurations are uniformly sampled in 3D space. To address this issue, we propose a novel graph-based policy model, dubbed Heterogeneous Equivariant Policy (HEPi), utilizing $SE(3)$ equivariant message passing networks as the main backbone to exploit the geometric symmetry. In addition, by modeling explicit heterogeneity, HEPi can outperform Transformer-based and non-heterogeneous equivariant policies in terms of average returns, sample efficiency, and generalization to unseen objects. Our project page is available at https://thobotics.github.io/hepi.