Shared Control with Black Box Agents using Oracle Queries
This work addresses shared control in robotics by enabling query-based communication, offering incremental improvements in learning efficiency and policy quality.
The paper tackles the problem of robots learning to collaborate with humans in shared control by introducing the ability to query cooperating agents, using two oracle types and three heuristics for query timing. Empirical results on two environments demonstrate that querying improves control policy learning and highlights trade-offs between the heuristics.
Shared control problems involve a robot learning to collaborate with a human. When learning a shared control policy, short communication between the agents can often significantly reduce running times and improve the system's accuracy. We extend the shared control problem to include the ability to directly query a cooperating agent. We consider two types of potential responses to a query, namely oracles: one that can provide the learner with the best action they should take, even when that action might be myopically wrong, and one with a bounded knowledge limited to its part of the system. Given this additional information channel, this work further presents three heuristics for choosing when to query: reinforcement learning-based, utility-based, and entropy-based. These heuristics aim to reduce a system's overall learning cost. Empirical results on two environments show the benefits of querying to learn a better control policy and the tradeoffs between the proposed heuristics.