Robo360: A 3D Omnispective Multi-Material Robotic Manipulation Dataset
This dataset addresses the need for better 3D perception in robotics, though it is incremental as it builds on existing methods like dynamic NeRF.
The paper tackles the challenge of complex real-world robotic manipulation by presenting Robo360, a dataset with dense view coverage and diverse objects to enable high-quality 3D neural representation learning and facilitate research in object manipulation and physical world modeling.
Building robots that can automate labor-intensive tasks has long been the core motivation behind the advancements in computer vision and the robotics community. Recent interest in leveraging 3D algorithms, particularly neural fields, has led to advancements in robot perception and physical understanding in manipulation scenarios. However, the real world's complexity poses significant challenges. To tackle these challenges, we present Robo360, a dataset that features robotic manipulation with a dense view coverage, which enables high-quality 3D neural representation learning, and a diverse set of objects with various physical and optical properties and facilitates research in various object manipulation and physical world modeling tasks. We confirm the effectiveness of our dataset using existing dynamic NeRF and evaluate its potential in learning multi-view policies. We hope that Robo360 can open new research directions yet to be explored at the intersection of understanding the physical world in 3D and robot control.