Online Behavior Modification for Expressive User Control of RL-Trained Robots
This addresses the need for expressive user control in human-robot interaction, though it is incremental as it builds on existing RL and shared autonomy methods.
The paper tackles the problem of limited user control over RL-trained robots after deployment by introducing online behavior modification, allowing real-time adjustment of robot behavior features. In a user study with 23 participants, ACORD provided user-preferred control and expression comparable to Shared Autonomy while maintaining RL's autonomy and robustness.
Reinforcement Learning (RL) is an effective method for robots to learn tasks. However, in typical RL, end-users have little to no control over how the robot does the task after the robot has been deployed. To address this, we introduce the idea of online behavior modification, a paradigm in which users have control over behavior features of a robot in real time as it autonomously completes a task using an RL-trained policy. To show the value of this user-centered formulation for human-robot interaction, we present a behavior diversity based algorithm, Adjustable Control Of RL Dynamics (ACORD), and demonstrate its applicability to online behavior modification in simulation and a user study. In the study (n=23) users adjust the style of paintings as a robot traces a shape autonomously. We compare ACORD to RL and Shared Autonomy (SA), and show ACORD affords user-preferred levels of control and expression, comparable to SA, but with the potential for autonomous execution and robustness of RL.