Adversarial Imitation Learning from Video using a State Observer
This addresses sample inefficiency for researchers and practitioners in imitation learning, though it is incremental as it builds on existing adversarial imitation learning frameworks.
The paper tackles the problem of high sample complexity in imitation learning from video demonstrations by introducing VGAIfO-SO, a new algorithm that uses a self-supervised state observer to estimate lower-dimensional state representations from images, resulting in improved sample efficiency compared to other methods and sometimes achieving performance close to algorithms with privileged state access.
The imitation learning research community has recently made significant progress towards the goal of enabling artificial agents to imitate behaviors from video demonstrations alone. However, current state-of-the-art approaches developed for this problem exhibit high sample complexity due, in part, to the high-dimensional nature of video observations. Towards addressing this issue, we introduce here a new algorithm called Visual Generative Adversarial Imitation from Observation using a State Observer VGAIfO-SO. At its core, VGAIfO-SO seeks to address sample inefficiency using a novel, self-supervised state observer, which provides estimates of lower-dimensional proprioceptive state representations from high-dimensional images. We show experimentally in several continuous control environments that VGAIfO-SO is more sample efficient than other IfO algorithms at learning from video-only demonstrations and can sometimes even achieve performance close to the Generative Adversarial Imitation from Observation (GAIfO) algorithm that has privileged access to the demonstrator's proprioceptive state information.