Closed-loop Bayesian Semantic Data Fusion for Collaborative Human-Autonomy Target Search
This addresses the challenge of dynamic target search for autonomous systems by incorporating human semantic inputs, though it appears incremental as it builds on existing CPOMDP methods with a new formulation.
The paper tackled the problem of autonomous vehicles efficiently reacquiring and localizing mobile targets in large spaces by developing a collaborative human-machine sensing solution using CPOMDP planning to integrate imperfect sensor data and semantic natural language observations from humans, validated in real human-robot indoor target search scenarios.
In search applications, autonomous unmanned vehicles must be able to efficiently reacquire and localize mobile targets that can remain out of view for long periods of time in large spaces. As such, all available information sources must be actively leveraged -- including imprecise but readily available semantic observations provided by humans. To achieve this, this work develops and validates a novel collaborative human-machine sensing solution for dynamic target search. Our approach uses continuous partially observable Markov decision process (CPOMDP) planning to generate vehicle trajectories that optimally exploit imperfect detection data from onboard sensors, as well as semantic natural language observations that can be specifically requested from human sensors. The key innovation is a scalable hierarchical Gaussian mixture model formulation for efficiently solving CPOMDPs with semantic observations in continuous dynamic state spaces. The approach is demonstrated and validated with a real human-robot team engaged in dynamic indoor target search and capture scenarios on a custom testbed.