Learning rewards for robotic ultrasound scanning using probabilistic temporal ranking
This work addresses the challenge of learning from sub-optimal demonstrations in robotics, particularly for medical imaging applications like ultrasound scanning, though it appears incremental as it builds on existing reward inference strategies.
The paper tackles the problem of inferring reward functions from exploratory demonstrations for robotic tasks where the goal is unknown, proposing a probabilistic temporal ranking method that improves performance in autonomous ultrasound scanning and other goal-oriented learning tasks.
Informative path-planning is a well established approach to visual-servoing and active viewpoint selection in robotics, but typically assumes that a suitable cost function or goal state is known. This work considers the inverse problem, where the goal of the task is unknown, and a reward function needs to be inferred from exploratory example demonstrations provided by a demonstrator, for use in a downstream informative path-planning policy. Unfortunately, many existing reward inference strategies are unsuited to this class of problems, due to the exploratory nature of the demonstrations. In this paper, we propose an alternative approach to cope with the class of problems where these sub-optimal, exploratory demonstrations occur. We hypothesise that, in tasks which require discovery, successive states of any demonstration are progressively more likely to be associated with a higher reward, and use this hypothesis to generate time-based binary comparison outcomes and infer reward functions that support these ranks, under a probabilistic generative model. We formalise this \emph{probabilistic temporal ranking} approach and show that it improves upon existing approaches to perform reward inference for autonomous ultrasound scanning, a novel application of learning from demonstration in medical imaging while also being of value across a broad range of goal-oriented learning from demonstration tasks. \keywords{Visual servoing \and reward inference \and probabilistic temporal ranking