Smart Magnetic Microrobots Learn to Swim with Deep Reinforcement Learning
This work addresses the challenge of precise microrobot control in biomimetic fluidic environments, which is incremental as it applies an existing reinforcement learning method to a specific domain.
The researchers tackled the problem of controlling swimming microrobots in complex, uncharacterized environments by using deep reinforcement learning to autonomously develop robust control policies, achieving successful learning with fewer than 100,000 training steps and enabling fine-tuning through regression.
Swimming microrobots are increasingly developed with complex materials and dynamic shapes and are expected to operate in complex environments in which the system dynamics are difficult to model and positional control of the microrobot is not straightforward to achieve. Deep reinforcement learning is a promising method of autonomously developing robust controllers for creating smart microrobots, which can adapt their behavior to operate in uncharacterized environments without the need to model the system dynamics. Here, we report the development of a smart helical magnetic hydrogel microrobot that used the soft actor critic reinforcement learning algorithm to autonomously derive a control policy which allowed the microrobot to swim through an uncharacterized biomimetic fluidic environment under control of a time varying magnetic field generated from a three-axis array of electromagnets. The reinforcement learning agent learned successful control policies with fewer than 100,000 training steps, demonstrating sample efficiency for fast learning. We also demonstrate that we can fine tune the control policies learned by the reinforcement learning agent by fitting mathematical functions to the learned policy's action distribution via regression. Deep reinforcement learning applied to microrobot control is likely to significantly expand the capabilities of the next generation of microrobots.