Modifying RL Policies with Imagined Actions: How Predictable Policies Can Enable Users to Perform Novel Tasks
This work addresses the challenge of user-robot collaboration in novel tasks for robotics applications, but it appears incremental as it builds on existing RL and teleoperation methods.
The paper tackles the problem of enabling users to creatively control robots with RL policies by addressing issues where user teleoperation leads to failure states, and presents the IODA algorithm to empower users in accomplishing new tasks.
It is crucial that users are empowered to use the functionalities of a robot to creatively solve problems on the fly. A user who has access to a Reinforcement Learning (RL) based robot may want to use the robot's autonomy and their knowledge of its behavior to complete new tasks. One way is for the user to take control of some of the robot's action space through teleoperation while the RL policy simultaneously controls the rest. However, an out-of-the-box RL policy may not readily facilitate this. For example, a user's control may bring the robot into a failure state from the policy's perspective, causing it to act in a way the user is not familiar with, hindering the success of the user's desired task. In this work, we formalize this problem and present Imaginary Out-of-Distribution Actions, IODA, an initial algorithm for addressing that problem and empowering user's to leverage their expectation of a robot's behavior to accomplish new tasks.