Planning with Pixels in (Almost) Real Time
This addresses the challenge of real-time decision-making in video games for AI systems, though it is incremental as it adapts existing planning methods to visual inputs.
The paper tackled the problem of planning in Atari 2600 games using pixel inputs instead of RAM states, showing that the planning approach achieves scores comparable to humans and learning methods without training, and an episodic rollout version enables near real-time performance.
Recently, width-based planning methods have been shown to yield state-of-the-art results in the Atari 2600 video games. For this, the states were associated with the (RAM) memory states of the simulator. In this work, we consider the same planning problem but using the screen instead. By using the same visual inputs, the planning results can be compared with those of humans and learning methods. We show that the planning approach, out of the box and without training, results in scores that compare well with those obtained by humans and learning methods, and moreover, by developing an episodic, rollout version of the IW(k) algorithm, we show that such scores can be obtained in almost real time.