Improved Memories Learning
This work addresses the complexity of reinforcement learning for researchers by offering a simpler, interpolation-based approach, though it appears incremental as it builds on existing supervised learning techniques.
The paper tackles the challenge of simplifying reinforcement learning by converting it into a supervised learning problem, proposing Improved Memories Learning (IMeL) to use neural networks for interpolation rather than encoding problem structure, with preliminary results presented as a baseline method.
We propose Improved Memories Learning (IMeL), a novel algorithm that turns reinforcement learning (RL) into a supervised learning (SL) problem and delimits the role of neural networks (NN) to interpolation. IMeL consists of two components. The first is a reservoir of experiences. Each experience is updated based on a non-parametric procedural improvement of the policy, computed as a bounded one-sample Monte Carlo estimate. The second is a NN regressor, which receives as input improved experiences from the reservoir (context points) and computes the policy by interpolation. The NN learns to measure the similarity between states in order to compute long-term forecasts by averaging experiences, rather than by encoding the problem structure in the NN parameters. We present preliminary results and propose IMeL as a baseline method for assessing the merits of more complex models and inductive biases.