Imagination-Augmented Agents for Deep Reinforcement Learning
This work addresses the problem of data inefficiency and model misspecification in reinforcement learning for AI agents, representing an incremental advancement by integrating and learning from model-based predictions in a novel way.
The paper tackled the challenge of improving data efficiency and robustness in deep reinforcement learning by introducing Imagination-Augmented Agents (I2As), which combine model-free and model-based approaches to learn implicit plans from environment predictions, resulting in enhanced performance and reduced sensitivity to model errors compared to baseline methods.
We introduce Imagination-Augmented Agents (I2As), a novel architecture for deep reinforcement learning combining model-free and model-based aspects. In contrast to most existing model-based reinforcement learning and planning methods, which prescribe how a model should be used to arrive at a policy, I2As learn to interpret predictions from a learned environment model to construct implicit plans in arbitrary ways, by using the predictions as additional context in deep policy networks. I2As show improved data efficiency, performance, and robustness to model misspecification compared to several baselines.