3D AffordanceNet: A Benchmark for Visual Object Affordance Understanding
This provides a foundational benchmark for 3D visual affordance understanding, essential for vision-guided robotics, though it is incremental as it extends existing 2D/2.5D work to 3D.
The authors introduced 3D AffordanceNet, a dataset of 23k shapes across 23 object categories annotated with 18 affordance categories, to address the lack of 3D visual affordance understanding for robotics, and evaluated three state-of-the-art networks on tasks like full-shape and partial-view estimation, showing it as a promising but challenging benchmark.
The ability to understand the ways to interact with objects from visual cues, a.k.a. visual affordance, is essential to vision-guided robotic research. This involves categorizing, segmenting and reasoning of visual affordance. Relevant studies in 2D and 2.5D image domains have been made previously, however, a truly functional understanding of object affordance requires learning and prediction in the 3D physical domain, which is still absent in the community. In this work, we present a 3D AffordanceNet dataset, a benchmark of 23k shapes from 23 semantic object categories, annotated with 18 visual affordance categories. Based on this dataset, we provide three benchmarking tasks for evaluating visual affordance understanding, including full-shape, partial-view and rotation-invariant affordance estimations. Three state-of-the-art point cloud deep learning networks are evaluated on all tasks. In addition we also investigate a semi-supervised learning setup to explore the possibility to benefit from unlabeled data. Comprehensive results on our contributed dataset show the promise of visual affordance understanding as a valuable yet challenging benchmark.