Continuous Mean-Zero Disagreement-Regularized Imitation Learning (CMZ-DRIL)
This addresses the challenge of data efficiency in imitation learning for robotics and simulation tasks, though it appears incremental as it builds on existing ensemble and regularization techniques.
The paper tackles the problem of imitation learning with limited expert demonstrations by introducing CMZ-DRIL, a method that uses a novel reward structure based on agent ensemble disagreement to improve performance without environment-specific rewards. Results show that CMZ-DRIL generates agents that behave more similarly to experts than previous approaches in waypoint-navigation and MuJoCo environments.
Machine-learning paradigms such as imitation learning and reinforcement learning can generate highly performant agents in a variety of complex environments. However, commonly used methods require large quantities of data and/or a known reward function. This paper presents a method called Continuous Mean-Zero Disagreement-Regularized Imitation Learning (CMZ-DRIL) that employs a novel reward structure to improve the performance of imitation-learning agents that have access to only a handful of expert demonstrations. CMZ-DRIL uses reinforcement learning to minimize uncertainty among an ensemble of agents trained to model the expert demonstrations. This method does not use any environment-specific rewards, but creates a continuous and mean-zero reward function from the action disagreement of the agent ensemble. As demonstrated in a waypoint-navigation environment and in two MuJoCo environments, CMZ-DRIL can generate performant agents that behave more similarly to the expert than primary previous approaches in several key metrics.