Fast Reinforcement Learning with Incremental Gaussian Mixture Models
This work addresses data efficiency for reinforcement learning practitioners, but it appears incremental as it builds on existing function approximation methods.
The paper tackles the problem of data-inefficient reinforcement learning in continuous state spaces by integrating an incremental Gaussian mixture network as a function approximator, resulting in a concise algorithm that learns from very few environment interactions.
This work presents a novel algorithm that integrates a data-efficient function approximator with reinforcement learning in continuous state spaces. An online and incremental algorithm capable of learning from a single pass through data, called Incremental Gaussian Mixture Network (IGMN), was employed as a sample-efficient function approximator for the joint state and Q-values space, all in a single model, resulting in a concise and data-efficient algorithm, i.e., a reinforcement learning algorithm that learns from very few interactions with the environment. Results are analyzed to explain the properties of the obtained algorithm, and it is observed that the use of the IGMN function approximator brings some important advantages to reinforcement learning in relation to conventional neural networks trained by gradient descent methods.