New research finds that the visual cortex – a part of the brain that receives and processes information from the eyes – also processes sound information, which can create visual imagery.

Led by Lars Muckli, a professor in the Institute of Neuroscience and Psychology at the University of Glasgow in the UK, the team reports its findings in the journal Current Biology.

The authors suggest processing auditory information enables the human visual system to predict incoming information, thus giving us a survival advantage, as Prof. Muckli explains:

“Sounds create visual imagery, mental images, and automatic projections. So, for example, if you are in a street and you hear the sound of an approaching motorbike, you expect to see a motorbike coming around the corner. If it turned out to be a horse, you’d be very surprised.”

In their study report the team explains that the early visual cortex in humans was traditionally thought to process simple visual information such as orientation, contrast, and spatial frequency – relayed from the retina. It then feeds this information forward to processes further along the visual system.

However, the idea that the early visual cortex also receives “non-retinal” information has not been sufficiently investigated, say the authors, despite the fact that feedback connections from other parts of the brain “greatly outnumber feedforward connections.”

For their study, they carried out five experiments using functional magnetic resonance imaging to observe early visual cortex activity in 10 volunteers.

In one experiment, they observed what happened as blindfolded volunteers listened to three natural sounds: birdsong, traffic noise and a crowd of people talking.

They used a special algorithm that interprets different patterns of brain activity to show that different categories of sound were being processed in the early visual cortex, in a similar way to categories of imagery.

“We show that category-specific information from both complex natural sounds and imagery can be read out from early visual cortex activity in blindfolded participants,” they note.

illustration depicting brain activityShare on Pinterest
The researchers found that the early visual cortex also receives information related to auditory perception, as well as visual input.

In another experiment, they showed how even imagined images – occurring in the absence of sight and sound input – gave rise to early visual cortex activity.

The authors conclude their experiments show “that early visual cortex receives non-retinal input from other brain areas when it is generated by auditory perception and/or imagery, and this input carries common abstract information.”

Prof. Muckli says the findings help us better understand how interconnected different regions of the brain are.

“The early visual cortex hasn’t previously been known to process auditory information, and while there is some anatomical evidence of interconnectedness in monkeys, our study is the first to clearly show a relationship in humans,” he adds.

While they now need to prove it, the team suspects the auditory information provides predictions that help the visual system focus on unexpected or surprising events. Such an ability offers a clear survival advantage.

“This might provide insights into mental health conditions such as schizophrenia or autism and help us understand how sensory perceptions differ in these individuals,” says Prof. Muckli.

The team plans to find out how precise predictive coding in the brain can be, by experimenting with a wider range of different sounds.

Funds from the European Research Council and the Biotechnology and Biological Sciences Research Council helped finance the study.

Medical News Today recently came across further evidence of the brain’s interconnectedness – this time of the visual system affecting the sound processing system – when a study published in the journal Neuron found short stays in darkness can boost hearing. There, the researchers suggested preventing sight for as little as a week may be enough to help the brain process sound more effectively.