Leveraging Explainability for Comprehending Referring Expressions in the Real World
This addresses the challenge of human-robot collaboration in uncontrolled environments by enabling robots to handle a wider range of object categories, though it is incremental as it builds on existing explainability techniques.
The paper tackles the problem of robots understanding ambiguous object descriptions in real-world scenes by proposing a method that leverages explainability to focus on active regions, enabling better disambiguation without relying on predefined object categories. The method outperforms a state-of-the-art baseline in scenes with ambiguous objects that cannot be recognized by existing detectors.
For effective human-robot collaboration, it is crucial for robots to understand requests from users and ask reasonable follow-up questions when there are ambiguities. While comprehending the users' object descriptions in the requests, existing studies have focused on this challenge for limited object categories that can be detected or localized with existing object detection and localization modules. On the other hand, in the wild, it is impossible to limit the object categories that can be encountered during the interaction. To understand described objects and resolve ambiguities in the wild, for the first time, we suggest a method by leveraging explainability. Our method focuses on the active regions of a scene to find the described objects without putting the previous constraints on object categories and natural language instructions. We evaluate our method in varied real-world images and observe that the regions suggested by our method can help resolve ambiguities. When we compare our method with a state-of-the-art baseline, we show that our method performs better in scenes with ambiguous objects which cannot be recognized by existing object detectors.