Coordinated Response Modulations Enable Flexible Use of Visual Information

AbstractWe use sensory information in remarkably flexible ways. We can generalize by ignoring task-irrelevant features, report different features of a stimulus, and use different actions to report a perceptual judgment. These forms of flexible behavior are associated with small modulations of the responses of sensory neurons. While the existence of these response modulations is indisputable, efforts to understand their function have been largely relegated to theory, where they have been posited to change information coding or enable downstream neurons to read out different visual and cognitive information using flexible weights. Here, we tested these ideas using a rich, flexible behavioral paradigm, multi-neuron, multi-area recordings in primary visual cortex (V1) and mid-level visual area V4. We discovered that those response modulations in V4 (but not V1) contain the ingredients necessary to enable flexible behavior, but not via those previously hypothesized mechanisms. Instead, we demonstrated that these response modulations are precisely coordinated across the population such that downstream neurons have ready access to the correct information to flexibly guide behavior without making changes to information coding or synapses. Our results suggest a novel computational role for task-dependent response modulations: they enable flexible behavior by changing the information that gets out of a sensory area, not by changing information coding within it.SignificanceNatural perceptual judgments are continuous, generalized, and flexible. We estimate the ripeness of a piece of fruit on a continuous scale, we generalize by judging the ripeness of either a mango or an avocado even though they look very different, we flexibly judge either the size or the ripeness of the same piece of fruit, and we can flexibly indicate the same perceptual judgment using a variety of behaviors such as by speaking or writing any of many languages. Here, we show that the response modulations in visual cortex long associated with cognitive processes, surround modulation, or motor planning are sufficient to guide all these aspects of natural perceptual decision-making. We find that across the population, these response modulations reorient and reformat visual representations so that the relevant information is used to guide behavior via communication with downstream neurons. Our results are an example of a general computational principle for flexible behavior that emerges from the coordinated activity of large populations of neurons.