Real-Time Shadow Depth Cue for Touchless Hand Gesture on a Desktop Virtual Reality: Effective Control using fLight Model

With the advent of affordable hand gesture sensors and intriguing interaction techniques like touchless hand gesture interaction, a plethora of virtual reality (VR) learning applications have been developed recently with the aim of being user-friendly. In computer vision, image depth is particularly crucial when imitating real-world object in virtual environment (VE). As touchless hand gestures enable users to communicate without physically touching objects, creating a precise interaction in VE, particularly in controlling objects effectively, such as grasping, moving and locating, under varied illumination, is still challenging. Moreover, visual cues for touchless hand movements are crucial. Nonetheless, inappropriate visual cues produced by inadequate illumination, such as real-time shadow depth cues, however, do hinder users' ability to reach and grasp objects due to difficulty in estimating the object's 3D hand position over the object. Even after several modifications, the user was still having trouble grasping objects and, in fact, the object falls easily when moving it. Further, the users do experience difficulty to locate the object. Therefore, this study investigates the use of the fLight illumination model to improve real-time shadow depth cues in touchless hand gesture interactions using the Leap Motion Controller (LMC) in a desktop VR (dVR) environment. A series of iterative experiments were conducted to acquire qualitative and quantitative findings. Results indicate that precise real-time shadow depth cues significantly enhance user control in grasping, moving and locating virtual objects under different ambient physical illumination, instead of improve the visibility.