HyperPPO: A scalable method for finding small policies for robotic control
This addresses the need for memory-efficient models in robotics, offering a scalable method to generate multiple trained policies for users with computational constraints.
The paper tackles the problem of finding small neural network architectures for robotic control by proposing HyperPPO, an on-policy reinforcement learning algorithm that uses graph hypernetworks to estimate weights for multiple architectures simultaneously, resulting in highly performant policies for decentralized control of a Crazyflie2.1 quadrotor.
Models with fewer parameters are necessary for the neural control of memory-limited, performant robots. Finding these smaller neural network architectures can be time-consuming. We propose HyperPPO, an on-policy reinforcement learning algorithm that utilizes graph hypernetworks to estimate the weights of multiple neural architectures simultaneously. Our method estimates weights for networks that are much smaller than those in common-use networks yet encode highly performant policies. We obtain multiple trained policies at the same time while maintaining sample efficiency and provide the user the choice of picking a network architecture that satisfies their computational constraints. We show that our method scales well - more training resources produce faster convergence to higher-performing architectures. We demonstrate that the neural policies estimated by HyperPPO are capable of decentralized control of a Crazyflie2.1 quadrotor. Website: https://sites.google.com/usc.edu/hyperppo