Simple, Good, Fast: Self-Supervised World Models Free of Baggage
This work addresses the need for efficient world models in reinforcement learning, though it appears incremental by building on existing self-supervised methods.
The paper tackled the problem of simplifying world models by removing components like RNNs and image reconstructions, introducing SGF, which achieved good performance on the Atari 100k benchmark.
What are the essential components of world models? How far do we get with world models that are not employing RNNs, transformers, discrete representations, and image reconstructions? This paper introduces SGF, a Simple, Good, and Fast world model that uses self-supervised representation learning, captures short-time dependencies through frame and action stacking, and enhances robustness against model errors through data augmentation. We extensively discuss SGF's connections to established world models, evaluate the building blocks in ablation studies, and demonstrate good performance through quantitative comparisons on the Atari 100k benchmark.