Planning from Pixels using Inverse Dynamics Models
This work is significant for model-based reinforcement learning agents struggling with dynamics modeling in high-dimensional visual environments, offering a way to improve planning with sparse rewards.
This paper addresses the challenge of learning task-agnostic dynamics models in high-dimensional observation spaces for model-based reinforcement learning. They propose learning latent world models by predicting future action sequences conditioned on task completion, which improves performance on visual goal completion tasks compared to prior model-free approaches.
Learning task-agnostic dynamics models in high-dimensional observation spaces can be challenging for model-based RL agents. We propose a novel way to learn latent world models by learning to predict sequences of future actions conditioned on task completion. These task-conditioned models adaptively focus modeling capacity on task-relevant dynamics, while simultaneously serving as an effective heuristic for planning with sparse rewards. We evaluate our method on challenging visual goal completion tasks and show a substantial increase in performance compared to prior model-free approaches.