Review on vision‐based tracking in surgical navigation

Computer vision is an important cornerstone for the foundation of many modern technologies. The development of modern computer‐aided‐surgery, especially in the context of surgical navigation for minimally invasive surgery, is one example. Surgical navigation provides the necessary spatial information in computer‐aided‐surgery. Amongst the various forms of perception, vision‐based sensing has been proposed as a promising candidate for tracking and localisation application largely due to its ability to provide timely intra‐operative feedback and contactless sensing. The motivation for vision‐based sensing in surgical navigation stems from many factors, including the challenges faced by other forms of navigation systems. A common surgical navigation system performs tracking of surgical tools with external tracking systems, which may suffer from both technical and usability issues. Vision‐based tracking offers a relatively streamlined framework compared to those approaches implemented with external tracking systems. This review study aims to discuss contemporary research and development in vision‐based sensing for surgical navigation. The selected review materials are expected to provide a comprehensive appreciation of state‐of‐the‐art technology and technical issues enabling holistic discussions of the challenges and knowledge gaps in contemporary development. Original views on the significance and development prospect of vision‐based sensing in surgical navigation are presented.