Learning a Multi-Modal Policy via Imitating Demonstrations with Mixed Behaviors
This addresses the challenge of learning diverse behaviors from unlabeled demonstrations for robotics or AI systems, though it is incremental as it builds on existing variational autoencoder frameworks.
The paper tackles the problem of training a multi-modal policy from mixed demonstrations without behavior labels by discovering latent factors of variation, and it shows competitive performance with methods using ground truth labels on three tasks, including one with high-dimensional visual inputs.
We propose a novel approach to train a multi-modal policy from mixed demonstrations without their behavior labels. We develop a method to discover the latent factors of variation in the demonstrations. Specifically, our method is based on the variational autoencoder with a categorical latent variable. The encoder infers discrete latent factors corresponding to different behaviors from demonstrations. The decoder, as a policy, performs the behaviors accordingly. Once learned, the policy is able to reproduce a specific behavior by simply conditioning on a categorical vector. We evaluate our method on three different tasks, including a challenging task with high-dimensional visual inputs. Experimental results show that our approach is better than various baseline methods and competitive with a multi-modal policy trained by ground truth behavior labels.