The what and where of synchronous sound perception

A bstract Sound localization and identification are challenging in acoustically rich environments. The relation between these two processes is still poorly understood. As natural sound-sources rarely occur exactly simultaneously, we wondered whether the auditory system could identify (“what”) and localize (“where”) two spatially separated sounds with synchronous onsets. While listeners typically report hearing a single source at an average location, one study found that both sounds may be accurately localized if listeners are explicitly being told two sources exist. We here tested whether simultaneous source identification (one vs. two) and localization is possible, by letting listeners choose to make either one or two head-orienting saccades to the perceived location(s). Results show that listeners could identify two sounds only when presented on different sides of the head, and that identification accuracy increased with their spatial separation. Notably, listeners were unable to accurately localize either sound, irrespective of whether one or two sounds were identified. Instead, the first (or only) response always landed near the average location, while second responses were unrelated to the targets. We conclude that localization of synchronous sounds in the absence of prior information is impossible. We discuss that the putative cortical ‘what’ pathway may not transmit relevant information to the ‘where’ pathway. We examine how a broadband interaural correlation cue could help to correctly identify the presence of two sounds without being able to localize them. We propose that the persistent averaging behavior reveals that the ‘where’ system intrinsically assumes that synchronous sounds originate from a single source. S ignificance S tatement It is poorly understood whether identification (‘what’) of sounds and their localization (‘where’) are inter-related, or independent neural processes. We measured sound-localization responses towards synchronous sounds to examine potential coupling of these processes. We varied the spatial configurations of two sounds and found that although identification improved considerably with larger spatial separation, their localization was unaffected: responses were always directed towards the average location. This shows absence of mutual coupling of information between the ‘what’ and ‘where’ streams in the auditory system. We also show how broadband interaural correlation could explain the improved identification results, without affecting localization performance, and explain how the persistent spatial averaging could be understood from strong internal priors regarding sound synchronicity.