A Stereo Vision-Based Method for Reconstructing 3-D Hand Motion in Real-Time

Hand motion paralysis negatively affects the lives of the involved patients. To recover the hand motions into their normal condition, these patients need to be taken into complex and long-term rehabilitation treatments. During rehabilitation, hand motions with full finger features need to be tracked accurately in 3-D dimension in real-time for analyzing and diagnosing hand motion paralysis. However, most studies tried to track hand motions based on contact sensors. These methods are not user-friendly. Even using contactless sensors, most of them could only detect the hand motions in 2-D image spaces. Consequently, in this study, we developed a stereo vision-based method for detecting and tracking 3-D hand features in real-time. In particular, we employed a convolutional deep neural network (C-DNN) for tracking by detecting hand-finger features. The features were tracked on left and right images captured by a stereo camera system before being reconstructed into 3-D spaces. A meta-validation procedure was conducted to compute the accuracy of the method with various light conditions, skin colors, and hand shapes. As a result, we could successfully track hand motions in real-time with acceptable accuracy. In perspective, we will apply the method for analyzing and diagnosing hand paralysis inside a clinical decision-support system.