Modelling the role of contour integration in visual inference

Abstract Under difficult viewing conditions, the brain’s visual system uses a variety of recurrent modulatory mechanisms to augment feed-forward processing. One resulting phenomenon is contour integration, which occurs in the primary visual (V1) cortex and strengthens neural responses to edges if they belong to a larger smooth contour. Computational models have contributed to an understanding of the circuit mechanisms of contour integration, but less is known about its role in visual perception. To address this gap, we embedded a biologically grounded model of contour integration in a task-driven artificial neural network, and trained it using a gradient-descent variant. We used this model to explore how brain-like contour integration may be optimized for high-level visual objectives as well as its potential roles in perception. When the model was trained to detect contours in a background of random edges, a task commonly used to examine contour integration in the brain, it closely mirrored the brain in terms of behavior, neural responses, and lateral connection patterns. When trained on natural images, the model enhanced weaker contours and distinguished whether two points lay on the same vs. different contours. The model learnt robust features that generalized well to out-of-training-distribution stimuli. Surprisingly, and in contrast with the synthetic task, a parameter-matched control network without recurrence performed the same or better than the model on the natural-image tasks. Thus a contour integration mechanism is not essential to perform these more naturalistic contour-related tasks. Finally, the best performance in all tasks was achieved by a modified contour integration model that did not distinguish between excitatory and inhibitory neurons. Author summary Deep networks are machine-learning systems that consist of interconnected neuron-like elements. More than other kinds of artificial system, they rival human information processing in a variety of tasks. These structural and functional parallels have raised interest in using deep networks as simplified models of the brain, to better understand of brain function. For example, incorporating additional biological phenomena into deep networks may help to clarify how they affect brain function. In this direction, we adapted a deep network to incorporate a model of visual contour integration, a process in the brain that makes contours appear more visually prominent. We found that suitable training led this model to behave much like the corresponding brain circuits. We then investigated potential roles of the contour integration mechanism in processing of natural images, an important question that has been difficult to answer. The results were not straightforward. For example, the contour integration mechanism actually impaired the network’s ability to tell whether two points lay on the same contour or not, but improved the network’s ability to generalize this skill to a different group of images. Overall, this approach has raised more sophisticated questions about the role of contour integration in natural vision.