Javascript must be enabled to continue!
STTG-net: a Spatio-temporal network for human motion prediction based on transformer and graph convolution network
View through CrossRef
AbstractIn recent years, human motion prediction has become an active research topic in computer vision. However, owing to the complexity and stochastic nature of human motion, it remains a challenging problem. In previous works, human motion prediction has always been treated as a typical inter-sequence problem, and most works have aimed to capture the temporal dependence between successive frames. However, although these approaches focused on the effects of the temporal dimension, they rarely considered the correlation between different joints in space. Thus, the spatio-temporal coupling of human joints is considered, to propose a novel spatio-temporal network based on a transformer and a gragh convolutional network (GCN) (STTG-Net). The temporal transformer is used to capture the global temporal dependencies, and the spatial GCN module is used to establish local spatial correlations between the joints for each frame. To overcome the problems of error accumulation and discontinuity in the motion prediction, a revision method based on fusion strategy is also proposed, in which the current prediction frame is fused with the previous frame. The experimental results show that the proposed prediction method has less prediction error and the prediction motion is smoother than previous prediction methods. The effectiveness of the proposed method is also demonstrated comparing it with the state-of-the-art method on the Human3.6 M dataset.
Springer Science and Business Media LLC
Title: STTG-net: a Spatio-temporal network for human motion prediction based on transformer and graph convolution network
Description:
AbstractIn recent years, human motion prediction has become an active research topic in computer vision.
However, owing to the complexity and stochastic nature of human motion, it remains a challenging problem.
In previous works, human motion prediction has always been treated as a typical inter-sequence problem, and most works have aimed to capture the temporal dependence between successive frames.
However, although these approaches focused on the effects of the temporal dimension, they rarely considered the correlation between different joints in space.
Thus, the spatio-temporal coupling of human joints is considered, to propose a novel spatio-temporal network based on a transformer and a gragh convolutional network (GCN) (STTG-Net).
The temporal transformer is used to capture the global temporal dependencies, and the spatial GCN module is used to establish local spatial correlations between the joints for each frame.
To overcome the problems of error accumulation and discontinuity in the motion prediction, a revision method based on fusion strategy is also proposed, in which the current prediction frame is fused with the previous frame.
The experimental results show that the proposed prediction method has less prediction error and the prediction motion is smoother than previous prediction methods.
The effectiveness of the proposed method is also demonstrated comparing it with the state-of-the-art method on the Human3.
6 M dataset.
Related Results
Twilight graphs
Twilight graphs
AbstractThis paper deals primarily with countable, simple, connected graphs and the following two conditions which are trivially satisfied if the graphs are finite:(a) there is an ...
Categorizing Motion: Story-Based Categorizations
Categorizing Motion: Story-Based Categorizations
Our most primary goal is to provide a motion categorization for moving entities. A motion categorization that is related to how humans categorize motion, i.e., that is cognitive ...
Addressing the Limitations of Graph Neural Networks on Node-level Tasks
Addressing the Limitations of Graph Neural Networks on Node-level Tasks
As a generic data structure, graph is capable of modeling complex relations among objects in many real-world problems. Integrated with deep learning and graph signal processing, Gr...
Burning, edge burning & chromatic burning classification of some graph family
Burning, edge burning & chromatic burning classification of some graph family
Graph ‘G’ is a Simple and undirected graph, which has a lowest number of color that is required to color the edge is called chromatic index. It is denoted by the symbol χ1(G). In ...
Comparison of prospective and retrospective motion correction for Magnetic Resonance Imaging of the brain - Master's Thesis in Physics
Comparison of prospective and retrospective motion correction for Magnetic Resonance Imaging of the brain - Master's Thesis in Physics
Head motion is one of the most common sources of artefacts for Magnetic Resonance Imaging (MRI) of the brain. Especially children, being intimidated by the dimensions and the noise...
Diabot: A Predictive Medical Chatbot using Ensemble Learning
Diabot: A Predictive Medical Chatbot using Ensemble Learning
Accessibility to medical knowledge and healthcare costs are the two major impediments for common man. Conversational agents like Medical chatbots, which are designed keeping in vie...
Query driven-graph neural networks for community search
Query driven-graph neural networks for community search
Given one or more query vertices, Community Search (CS) aims to find densely intra-connected and loosely inter-connected structures containing query vertices. Attributed Community ...
Flutter prediction on combined EPS and carbon sandwich structure for light aircraft wing
Flutter prediction on combined EPS and carbon sandwich structure for light aircraft wing
Flutter prediction is an important step before conducting a flight test. In this study, we performed flutter prediction of a half-wing structure without control surfaces. The half-...