CNN-based Visual Servoing for Pose Control of Soft Fabric Parts

Visual servoing is one of the techniques to control robot pose based on captured images in real time. CNN-based visual servoing has been also proposed for positioning a robot end-effector with a hand-eye camera, a camera-mounted drone, and for positioning a rigid object grasped by a single manipulator. This paper proposes a new CNN-based visual servoing for positioning soft fabric parts by a dual manipulator system. The proposed control system aims to position a soft fabric part to the target pose while keeping the surface flat. Since the surface shape of the non-textured fabric part is difficult to recognize by a single grayscale image, we applied structured lighting to emphasize the features of the surface of the fabric part. Experiments demonstrate the fabric part positioning using a dual manipulator system and show that the fabric part can be positioned to the target pose with its surface kept flat.