Dream to Adapt: Meta Reinforcement Learning by Latent Context Imagination and MDP Imagination
This addresses data inefficiency in meta reinforcement learning for AI systems, though it appears incremental as it builds on existing context-based methods.
The paper tackles the problem of meta reinforcement learning requiring dense task coverage and large data by proposing MetaDreamer, which uses latent context and MDP imagination to reduce real training tasks and data, achieving improved data efficiency and interpolated generalization in benchmarks.
Meta reinforcement learning (Meta RL) has been amply explored to quickly learn an unseen task by transferring previously learned knowledge from similar tasks. However, most state-of-the-art algorithms require the meta-training tasks to have a dense coverage on the task distribution and a great amount of data for each of them. In this paper, we propose MetaDreamer, a context-based Meta RL algorithm that requires less real training tasks and data by doing meta-imagination and MDP-imagination. We perform meta-imagination by interpolating on the learned latent context space with disentangled properties, as well as MDP-imagination through the generative world model where physical knowledge is added to plain VAE networks. Our experiments with various benchmarks show that MetaDreamer outperforms existing approaches in data efficiency and interpolated generalization.