Tennis motion recognition. Design of classification approaches and experimental studies

This monograph presents studies concerning the challenging topic of Human Action Recognition, which has recently become of increasing importance to those working in the field of Computer Vision. It contains an overview of the research literature on tennis movements recognition based on video, images, motion capture data, and data registered using various sensors. In this monograph, three novel methods of tennis movements recognition are presented in detail with success, achieving high performance of the proposed classifiers. They all utilize Graph Convolutional Neural networks (GCN) solutions. Two methods are also applied using attention modules which improve feature extraction and thus make the classification better. The Spatial- -Temporal Graph Convolutional Network, the Attention-Temporal Graph Convolutional Network and the Dual Attention Temporal Graph Convolutional Network are applied for recognition of the most frequently used tennis moves, such as: forehand, backhand, volley forehand, and volley backhand. Moreover, the forehand and backhand are further divided into two phases: 1) the preparation, starting from the player’s positioning until the moment just before the ball makes contact with the racket, 2) the hit together with the racket swinging until the move is finished. These separations allow detailed analysis. For the purpose of this study, a 3DTennisDS dataset was created that contains motion capture data of the forehand, backhand, volley forehand, and volley backhand. They were registered via the Vicon motion capture system. Three proposed neural networks methods were verified based on the gathered dataset. In this monograph, the interpretability aspect of GCN was also presented. This issue is important and was examined by presenting feature maps and heatmaps obtained after selected convolutional operations. These results showed how individual filters in GCNs extract features that are relevant for recognizing tennis movements.