Augmentation of self-motion perception with synthetic auditory cues

Abstract People who suffer from vestibular loss or damage have difficulty maintaining balance and perceiving their own motion in space (self-motion). Sensory augmentation of vestibular information, via other senses, could improve these functions. Here, we tested whether synthetic auditory signals carrying self-motion information can be integrated with vestibular cues. Twenty healthy participants experienced self-motion stimuli in a 3D motion simulator, comprising vestibular (inertial motion), and/or synthetic auditory “motion” cues. The auditory cues, presented via stereo headphones, comprised a series of beeps, with motion speed encoded by beep rate, and heading direction (in the horizontal plane) encoded by simulating the sound to emanate from that direction. In each trial, participants experienced a single-interval self-motion stimulus (vestibular, auditory or combined), and their task was to discriminate its heading direction (two-alternative forced-choice, right/left of straight ahead). A slight heading conflict (Δ = ±6°) was introduced in the combined-cue condition to measure empirical cue weights. Combined auditory-vestibular thresholds were significantly lower (improved) compared to vestibular alone (p < 0.001). But integration was suboptimal – vestibular cues were overweighted, and combined-cue thresholds were larger than predicted by Bayesian-optimal weighting. Interestingly, combined-cue thresholds were better predicted by the empirically observed weights. Thus, humans can integrate synthetic auditory cues with natural vestibular cues to improve self-motion perception. However, synthetic cues are underweighted. This suggests that weighting is determined not only by cue reliability but also by perceived relevance, with (unnatural) synthetic cues potentially considered less relevant to the task. Training might be required to achieve reliability-based integration of synthetic and natural cues.