Human-AI Shared Control via Policy Dissection
This addresses the challenge of integrating human input into AI-controlled systems for tasks like autonomous driving and robotics, offering a practical solution without redesigning training paradigms, though it is incremental in building on existing RL methods.
The paper tackles the problem of enabling human-AI shared control in complex environments by converting pre-trained reinforcement learning policies into human-interactive ones without retraining, using a frequency-based approach called Policy Dissection. The results show substantial improvements in performance and safety for autonomous driving in unseen traffic scenes and versatile controllable motion skills for locomotion robots.
Human-AI shared control allows human to interact and collaborate with AI to accomplish control tasks in complex environments. Previous Reinforcement Learning (RL) methods attempt the goal-conditioned design to achieve human-controllable policies at the cost of redesigning the reward function and training paradigm. Inspired by the neuroscience approach to investigate the motor cortex in primates, we develop a simple yet effective frequency-based approach called \textit{Policy Dissection} to align the intermediate representation of the learned neural controller with the kinematic attributes of the agent behavior. Without modifying the neural controller or retraining the model, the proposed approach can convert a given RL-trained policy into a human-interactive policy. We evaluate the proposed approach on the RL tasks of autonomous driving and locomotion. The experiments show that human-AI shared control achieved by Policy Dissection in driving task can substantially improve the performance and safety in unseen traffic scenes. With human in the loop, the locomotion robots also exhibit versatile controllable motion skills even though they are only trained to move forward. Our results suggest the promising direction of implementing human-AI shared autonomy through interpreting the learned representation of the autonomous agents. Demo video and code will be made available at https://metadriverse.github.io/policydissect.