Programmable Control of Ultrasound Swarmbots through Reinforcement Learning
This work addresses the challenge of enabling intelligent, autonomous navigation for microrobots in unstructured environments, which could advance minimally invasive therapeutic and diagnostic procedures, though it is incremental in applying reinforcement learning to an existing acoustic manipulation approach.
The researchers tackled the problem of autonomously controlling microbubble microrobots for targeted drug delivery by using reinforcement learning to learn their dynamics and manipulate them with acoustic forces, achieving the first demonstration of autonomous acoustic navigation in a microfluidic environment with over 100,000 training images and validating robust control.
Powered by acoustics, existing therapeutic and diagnostic procedures will become less invasive and new methods will become available that have never been available before. Acoustically driven microrobot navigation based on microbubbles is a promising approach for targeted drug delivery. Previous studies have used acoustic techniques to manipulate microbubbles in vitro and in vivo for the delivery of drugs using minimally invasive procedures. Even though many advanced capabilities and sophisticated control have been achieved for acoustically powered microrobots, there remain many challenges that remain to be solved. In order to develop the next generation of intelligent micro/nanorobots, it is highly desirable to conduct accurate identification of the micro-nanorobots and to control their dynamic motion autonomously. Here we use reinforcement learning control strategies to learn the microrobot dynamics and manipulate them through acoustic forces. The result demonstrated for the first time autonomous acoustic navigation of microbubbles in a microfluidic environment. Taking advantage of the benefit of the second radiation force, microbubbles swarm to form a large swarm, which is then driven along the desired trajectory. More than 100 thousand images were used for the training to study the unexpected dynamics of microbubbles. As a result of this work, the microrobots are validated to be controlled, illustrating a good level of robustness and providing computational intelligence to the microrobots, which enables them to navigate independently in an unstructured environment without requiring outside assistance.