Ultrasound Microrobots with Reinforcement Learning

Abstract Ultrasound is an attractive modality for controlling micro/nanorobots due to penetrating deep into tissue, not being affected by the opaque nature of animal bodies, and generating a broad range of forces. However, ultrasound microrobots have poor navigation capabilities and the number of parameters involved in its motion make it extremely challenging for a human controller to accurately predict and manually correct the microrobot's position in real time. Herein, reinforcement learning control strategy is implemented to learn microrobot dynamics by accurately identifying micro/nanorobots (object detection and tracking) and manipulating them with ultrasound. This work demonstrates autonomous navigation of ultrasound microrobots in a fluidic environment. The propulsion strategy relies on the combined action of the primary and secondary radiation forces. Microswarms are formed through the secondary acoustic radiation force, while the primary acoustic radiation force guides the microrobots along a desired trajectory. Microrobots are trained using more than 100 000 images to study their unexpected dynamics. The control of the microrobots is validated, illustrating a good level of robustness and providing the microrobots with computational intelligence that enables them to navigate independently in an unstructured environment without outside assistance.