Cross-subject Mapping of Neural Activity with Restricted Boltzmann Machines

Abstract Subject-to-subject variability is a common challenge in generalizing neural data models across subjects. While many methods exist that map one subject to another, it remains challenging to combine many subjects in a computationally efficient manner, especially with features that are highly non-linear such as when considering populations of spiking neurons or motor units. Our objective is to transfer data from one or more target subjects to the data space of one or more source subject(s) such that the neural decoder of the source subject can directly decode the target data when the source(s) is not available during test time. We propose to use the Gaussian-Bernoulli Restricted Boltzmann Machine (RBM); once trained over the entire set of subjects, the RBM allows the mapping of target features on source feature spaces using Gibbs sampling. We also consider a novel computationally efficient training technique for RBMs based on the minimization of the Fisher divergence, which allows the gradients of the RBM to be computed in closed form, in contrast to the more traditional contrastive divergence. We apply our methods to decode turning behaviors from a comprehensive spike-resolved motor program – neuromuscular recordings of spike trains from the ten muscles that control wing motion in an agile flying Manduca sexta. The dataset consists of the comprehensive motor program recorded from nine subjects driven by six discrete visual stimuli. The evaluations show that the target features can be decoded using the source classifier with an accuracy of up to 95% when mapped using an RBM trained by Fisher divergence. Significant Statement In this study, we address the variability of neural data across subjects, which is a significant obstacle in developing models that can generalize across subjects. Our objective is to create a task-specific representation of the target subject signal in the feature space of the source subject. Our proposed RBM architectures achieve highly flexible and accurate cross-subject mapping with few assumptions. Our Fisher RBM improved the previous state of the art method by 300%. Our methods show promise in generalizing features of complex neural datasets across individuals, tuning neural interfaces to subject-specific features, and leveraging data across multiple subjects when experiments are limited in time or completeness.