TASKOGRAPHY: Evaluating robot task planning over large 3D scene graphs
This addresses a bottleneck in robotics for researchers and practitioners by providing tools to make task planning over complex 3D environments more efficient, though it is incremental in improving existing methods rather than introducing a new paradigm.
The paper tackles the problem of robot task planning over large 3D scene graphs (3DSGs), which are impractical for real-time planning due to their size, and introduces TASKOGRAPHY as the first large-scale benchmark for this. It proposes SCRUB for sparsifying 3DSGs, enabling classical planners to match or surpass learning-based ones, and SEEK to help learning-based planners exploit 3DSG structure, reducing replanning queries by an order of magnitude.
3D scene graphs (3DSGs) are an emerging description; unifying symbolic, topological, and metric scene representations. However, typical 3DSGs contain hundreds of objects and symbols even for small environments; rendering task planning on the full graph impractical. We construct TASKOGRAPHY, the first large-scale robotic task planning benchmark over 3DSGs. While most benchmarking efforts in this area focus on vision-based planning, we systematically study symbolic planning, to decouple planning performance from visual representation learning. We observe that, among existing methods, neither classical nor learning-based planners are capable of real-time planning over full 3DSGs. Enabling real-time planning demands progress on both (a) sparsifying 3DSGs for tractable planning and (b) designing planners that better exploit 3DSG hierarchies. Towards the former goal, we propose SCRUB, a task-conditioned 3DSG sparsification method; enabling classical planners to match and in some cases surpass state-of-the-art learning-based planners. Towards the latter goal, we propose SEEK, a procedure enabling learning-based planners to exploit 3DSG structure, reducing the number of replanning queries required by current best approaches by an order of magnitude. We will open-source all code and baselines to spur further research along the intersections of robot task planning, learning and 3DSGs.