Binarized P-Network: Deep Reinforcement Learning of Robot Control from Raw Images on FPGA
This work addresses the problem of enabling efficient, image-based control for edge robots on FPGAs, representing an incremental advance by adapting existing methods to hardware constraints.
The paper tackled the challenge of implementing deep reinforcement learning for robot control on power-efficient FPGAs, which are ill-suited for real-number operations, by proposing a Binarized P-Network algorithm that uses binarized CNNs and conservative value iteration, achieving successful visual tracking in both simulation and real-robot experiments.
This paper explores a Deep Reinforcement Learning (DRL) approach for designing image-based control for edge robots to be implemented on Field Programmable Gate Arrays (FPGAs). Although FPGAs are more power-efficient than CPUs and GPUs, a typical DRL method cannot be applied since they are composed of many Logic Blocks (LBs) for high-speed logical operations but low-speed real-number operations. To cope with this problem, we propose a novel DRL algorithm called Binarized P-Network (BPN), which learns image-input control policies using Binarized Convolutional Neural Networks (BCNNs). To alleviate the instability of reinforcement learning caused by a BCNN with low function approximation accuracy, our BPN adopts a robust value update scheme called Conservative Value Iteration, which is tolerant of function approximation errors. We confirmed the BPN's effectiveness through applications to a visual tracking task in simulation and real-robot experiments with FPGA.