High Latency Unmanned Ground Vehicle Teleoperation Enhancement Through Video Transformation

Long-distance and high latency teleoperation tasks are difficult, highly stressful to the teleoperators, and prone to over-correction, which can lead to loss of control and damage to teleoperated vehicles. Beyond a certain latency, or at higher ground speeds during teleoperation, the situation can get significantly worse and ultimately lead to the teleoperation task being impossible to carry out. To overcome this problem, this research work investigates a 2D visual feedback-based assistive interface (sliding window and sliding with zooming window) that applies simple but effective video transformations to enhance teleoperation in high latency situations. A teleoperation simulator that can replicate a teleoperation scenario affected by high and adjustable latency, and that allows operators to control simulated ground vehicles with reasonable ground speed has been developed. PSNR, SSIM, and multi-SSIM have been used to fine-tune and optimise the assistive interfaces. A human survey was conducted to evaluate the teleoperation system and compare the performance of the assistive interfaces. The survey has shown that a 900ms delay increases the task completion time by up to 205% for the on-road and 147% for the off-road driving track. Further, the overcorrection-induced oscillations increase by up to 718%. The survey shows that our sliding window and sliding with zooming window enhances the teleoperation significantly. The sliding-only video transformation reduces the task completion time by up to 25.53%, and the sliding with zooming transformation reduces the task completion time by up to 21.82%. The sliding-only interface reduces the oscillation count by up to 66.28%, and the sliding with zooming interface reduces it by up to 75.58%. The qualitative feedback from the participants also shows that both types of assistive interfaces offer better visual situational awareness, comfort, and controllability, and significantly reduce the impact of latency and intermittency. <br>