Belief-Grounded Networks for Accelerated Robot Learning under Partial Observability
This addresses the problem of partial observability in robotics, offering a more efficient method for policy learning, though it appears incremental as it builds on existing belief and history-based approaches.
The paper tackles policy learning under partial observability in robotics by proposing Belief-Grounded Networks (BGN), which use an auxiliary belief-reconstruction loss to summarize input history, and it outperforms baselines on benchmark and novel robotic touch-sensing tasks, with successful transfer to a physical robot.
Many important robotics problems are partially observable in the sense that a single visual or force-feedback measurement is insufficient to reconstruct the state. Standard approaches involve learning a policy over beliefs or observation-action histories. However, both of these have drawbacks; it is expensive to track the belief online, and it is hard to learn policies directly over histories. We propose a method for policy learning under partial observability called the Belief-Grounded Network (BGN) in which an auxiliary belief-reconstruction loss incentivizes a neural network to concisely summarize its input history. Since the resulting policy is a function of the history rather than the belief, it can be executed easily at runtime. We compare BGN against several baselines on classic benchmark tasks as well as three novel robotic touch-sensing tasks. BGN outperforms all other tested methods and its learned policies work well when transferred onto a physical robot.