From marker features to multimodal fusion: a review of vision-based tactile sensor design and development

Abstract Vision-based tactile sensors capture visual information from contact surfaces to perceive tactile signals. In recent years, these sensors have been widely used in robotic systems to enhance their tactile perception capabilities. To meet the demands of various application scenarios, numerous hardware designs for these sensors have been developed. This paper reviews the design and development of vision-based tactile sensors. Based on their sensing principles and implementation methods, we categorize existing sensors into three main types: Vision-based tactile sensors based on marked features, vision-based tactile sensors based on coating geometric features, and vision-tactile modality fusion sensors. For each type, we delve into the core technical challenges, existing solutions, and corresponding hardware implementation strategies. By summarizing the characteristics and solutions of existing sensors, this paper aims to provide researchers with a comprehensive reference to past studies and solutions, while also exploring potential future research directions.