Open-Vocabulary Affordance Detection in 3D Point Clouds
This addresses the adaptability of intelligent robots in complex environments by allowing detection of unbounded affordances without annotations, though it is incremental as it builds on existing affordance detection methods.
The paper tackles the problem of affordance detection in 3D point clouds, which is limited by predefined labels, by proposing an open-vocabulary method that enables zero-shot detection of unseen affordances, outperforming baselines by a large margin with a fast inference speed of ~100ms.
Affordance detection is a challenging problem with a wide variety of robotic applications. Traditional affordance detection methods are limited to a predefined set of affordance labels, hence potentially restricting the adaptability of intelligent robots in complex and dynamic environments. In this paper, we present the Open-Vocabulary Affordance Detection (OpenAD) method, which is capable of detecting an unbounded number of affordances in 3D point clouds. By simultaneously learning the affordance text and the point feature, OpenAD successfully exploits the semantic relationships between affordances. Therefore, our proposed method enables zero-shot detection and can be able to detect previously unseen affordances without a single annotation example. Intensive experimental results show that OpenAD works effectively on a wide range of affordance detection setups and outperforms other baselines by a large margin. Additionally, we demonstrate the practicality of the proposed OpenAD in real-world robotic applications with a fast inference speed (~100ms). Our project is available at https://openad2023.github.io.