Interactive Natural Language-based Person Search
This addresses the problem of person search for robotics and surveillance applications, but it is incremental as it builds on existing models and methods.
The paper tackles the problem of searching for people in unconstrained environments using natural language descriptions, proposing an algorithm that adapts existing models for visual and language understanding and an iterative QA strategy to request additional information, achieving promising results validated on benchmark datasets and a mobile robot.
In this work, we consider the problem of searching people in an unconstrained environment, with natural language descriptions. Specifically, we study how to systematically design an algorithm to effectively acquire descriptions from humans. An algorithm is proposed by adapting models, used for visual and language understanding, to search a person of interest (POI) in a principled way, achieving promising results without the need to re-design another complicated model. We then investigate an iterative question-answering (QA) strategy that enable robots to request additional information about the POI's appearance from the user. To this end, we introduce a greedy algorithm to rank questions in terms of their significance, and equip the algorithm with the capability to dynamically adjust the length of human-robot interaction according to model's uncertainty. Our approach is validated not only on benchmark datasets but on a mobile robot, moving in a dynamic and crowded environment.