Two by Two: Learning Multi-Task Pairwise Objects Assembly for Generalizable Robot Manipulation
This addresses the challenge of generalizable robot manipulation for daily pairwise object assembly, which is incremental as it builds on existing shape assembly methods with a new dataset and method.
The paper tackles the problem of 3D assembly tasks for everyday objects by introducing the 2BY2 dataset with 1,034 instances across 18 tasks, and proposes a two-step SE(3) pose estimation method that achieves state-of-the-art performance on all tasks.
3D assembly tasks, such as furniture assembly and component fitting, play a crucial role in daily life and represent essential capabilities for future home robots. Existing benchmarks and datasets predominantly focus on assembling geometric fragments or factory parts, which fall short in addressing the complexities of everyday object interactions and assemblies. To bridge this gap, we present 2BY2, a large-scale annotated dataset for daily pairwise objects assembly, covering 18 fine-grained tasks that reflect real-life scenarios, such as plugging into sockets, arranging flowers in vases, and inserting bread into toasters. 2BY2 dataset includes 1,034 instances and 517 pairwise objects with pose and symmetry annotations, requiring approaches that align geometric shapes while accounting for functional and spatial relationships between objects. Leveraging the 2BY2 dataset, we propose a two-step SE(3) pose estimation method with equivariant features for assembly constraints. Compared to previous shape assembly methods, our approach achieves state-of-the-art performance across all 18 tasks in the 2BY2 dataset. Additionally, robot experiments further validate the reliability and generalization ability of our method for complex 3D assembly tasks.