Visual Sculpting: Visually-Aligned Planning Representations for Long-Horizon Robot Clay Sculpting
This work addresses the challenge of long-horizon planning for robotic sculpting of deformable materials, but the improvement over prior methods is incremental.
The paper tackles long-horizon robot clay sculpting by formulating it as a shape-to-shape matching problem and proposes a method that models deformable material dynamics in a visually-aligned representation capturing lighting and texture. The dynamics model achieves comparable performance to state-of-the-art while enabling visual planning for long-horizon tasks (>100 actions).
Clay sculpting is a nuanced, artistic task involving dexterous manipulation with long-horizon planning to achieve high-level goals. As a robotics problem, we formulate clay sculpting as a shape-to-shape matching challenge. Prior deformable object manipulation work either requires retraining a policy per goal or relies on dynamics models which represent state as sparse point clouds which do not capture important clay features, such as textures, well. We present a method for modeling the dynamics of deformable materials and planning for robotic sculpting in a representation that is visually-aligned, capturing lighting and texture features. With three different deformable materials and various end-effectors, we demonstrate that our dynamics model is comparable in performance to the state-of-the-art with the added benefit of being compatible with visual planning. Our actions are represented as parametrized pushes into clay with a single end-effector, which proved to be suitable for long-horizon (>100 actions) clay relief sculptures. Lastly, we show the benefits of planning in a visually-aligned representation, but also provide analysis providing evidence as to why this representation is challenging to plan in compared to 3D representations.