Optimal Neuron Selection: NK Echo State Networks for Reinforcement Learning
This addresses the challenge of efficient learning in recurrent networks for reinforcement learning tasks, though it appears incremental as it builds on existing Echo State Network concepts with a novel optimization approach.
The paper tackles the problem of learning in recurrent networks for reinforcement learning by introducing the NK Echo State Network, which reduces learning to optimizing an NK Landscape to select neurons without weight adjustment, and it achieves rapid learning and good generalization in balancing two poles on a cart without velocity information.
This paper introduces the NK Echo State Network. The problem of learning in the NK Echo State Network is reduced to the problem of optimizing a special form of a Spin Glass Problem known as an NK Landscape. No weight adjustment is used; all learning is accomplished by spinning up (turning on) or spinning down (turning off) neurons in order to find a combination of neurons that work together to achieve the desired computation. For special types of NK Landscapes, an exact global solution can be obtained in polynomial time using dynamic programming. The NK Echo State Network is applied to a reinforcement learning problem requiring a recurrent network: balancing two poles on a cart given no velocity information. Empirical results shows that the NK Echo State Network learns very rapidly and yields very good generalization.