Model-Free Episodic Control with State Aggregation
This incremental improvement makes episodic control more feasible for reinforcement learning tasks by addressing its practical limitations.
The paper tackled the high memory and computational demands of Model-Free Episodic Control in reinforcement learning by proposing a simple heuristic with state aggregation, resulting in reduced computational requirements without significant performance loss on Atari games when using conservative hyperparameters.
Episodic control provides a highly sample-efficient method for reinforcement learning while enforcing high memory and computational requirements. This work proposes a simple heuristic for reducing these requirements, and an application to Model-Free Episodic Control (MFEC) is presented. Experiments on Atari games show that this heuristic successfully reduces MFEC computational demands while producing no significant loss of performance when conservative choices of hyperparameters are used. Consequently, episodic control becomes a more feasible option when dealing with reinforcement learning tasks.