RoboBenchMart: Benchmarking Robots in Retail Environment
This addresses the problem of evaluating robots in realistic retail settings for researchers and developers, though it is incremental as it extends existing benchmarking approaches to a new domain.
The authors tackled the limitation of existing robotic manipulation benchmarks being too simplistic by introducing RoboBenchMart, a challenging benchmark for dark store retail environments with dense object clutter and varied spatial configurations, and demonstrated that current state-of-the-art generalist models struggle with common retail tasks.
Most existing robotic manipulation benchmarks focus on simplified tabletop scenarios, typically involving a stationary robotic arm interacting with various objects on a flat surface. To address this limitation, we introduce RoboBenchMart, a more challenging and realistic benchmark designed for dark store environments, where robots must perform complex manipulation tasks with diverse grocery items. This setting presents significant challenges, including dense object clutter and varied spatial configurations -- with items positioned at different heights, depths, and in close proximity. By targeting the retail domain, our benchmark addresses a setting with strong potential for near-term automation impact. We demonstrate that current state-of-the-art generalist models struggle to solve even common retail tasks. To support further research, we release the RoboBenchMart suite, which includes a procedural store layout generator, a trajectory generation pipeline, evaluation tools and fine-tuned baseline models.