Exploring Deep and Recurrent Architectures for Optimal Control
This work addresses the challenge of extending deep learning beyond perception to direct control tasks, representing an incremental step in applying neural networks to optimal control.
The paper tackled the problem of applying deep and recurrent neural networks to continuous, high-dimensional locomotion control by training neural network controllers with thousands of parameters using guided policy search, enabling comparisons of various architectures.
Sophisticated multilayer neural networks have achieved state of the art results on multiple supervised tasks. However, successful applications of such multilayer networks to control have so far been limited largely to the perception portion of the control pipeline. In this paper, we explore the application of deep and recurrent neural networks to a continuous, high-dimensional locomotion task, where the network is used to represent a control policy that maps the state of the system (represented by joint angles) directly to the torques at each joint. By using a recent reinforcement learning algorithm called guided policy search, we can successfully train neural network controllers with thousands of parameters, allowing us to compare a variety of architectures. We discuss the differences between the locomotion control task and previous supervised perception tasks, present experimental results comparing various architectures, and discuss future directions in the application of techniques from deep learning to the problem of optimal control.