Transferring Virtual Surgical Skills to Reality: AI Agents Mastering Surgical Decision Making in Vascular Interventional Robotics

Vascular intervention surgery offers advantages such as minimal invasiveness, quick recovery, and low side-effects. Achieving automatic guidewire navigation in interventional surgical robots can effectively assist doctors in performing the surgery. Achieving automated guidewire navigation for interventional surgical robots can effectively assist doctors in performing the surgery. Deep learning and reinforcement learning methods have been widely used for guidewire navigation tasks. However, there are challenges such as the simplicity of the simulated environment and the limited diversity of reward functions, which prevent the training results from demonstrating the intelligence required in more complex environments. Therefore, we propose a virtual training environment that incorporates real vascular projections to create a more complex environment. In this environment, we introduce the distance between the guidewire tip and the target point into the reward function, utilize real-time images as input states, employ a multi-threaded Proximal Policy Optimization (PPO) algorithm to accelerate convergence, and adopt a multi-stage training approach that divides the navigation task into multiple sub-tasks to reduce task difficulty. The results demonstrate the effectiveness of our method in achieving automated guidewire navigation in the virtual environment, improving the success rate of guidewire navigation, and enhancing the algorithm’s robustness. Finally, by visualizing the attention locations of the neural network on the incoming images in the virtual environment, we process real-time images from the physical environment and transfer the trained model from the virtual environment to our physical interventional surgical robot, thus validating the feasibility of our method in real-world scenarios.