Explaining the Effect of Likelihood Manipulation and Prior through a Neural Network of the Audiovisual Perception of Space

Ursino, Mauro; Cuppini, Cristiano; Magosso, Elisa; Beierholm, Ulrik; Shams, Ladan

doi:10.1163/22134808-20191324

Explaining the Effect of Likelihood Manipulation and Prior through a Neural Network of the Audiovisual Perception of Space

Ursino, Mauro; Cuppini, Cristiano; Magosso, Elisa; Beierholm, Ulrik; Shams, Ladan

Authors

Mauro Ursino

Cristiano Cuppini

Elisa Magosso

Dr Ulrik Beierholm ulrik.beierholm@durham.ac.uk
Associate Professor

Ladan Shams

Abstract

Results in the recent literature suggest that multisensory integration in the brain follows the rules of Bayesian inference. However, how neural circuits can realize such inference and how it can be learned from experience is still the subject of active research. The aim of this work is to use a recent neurocomputational model to investigate how the likelihood and prior can be encoded in synapses, and how they affect audio-visual perception, in a variety of conditions characterized by different experience, different cue reliabilities and temporal asynchrony. The model considers two unisensory networks (auditory and visual) with plastic receptive fields and plastic crossmodal synapses, trained during a learning period. During training visual and auditory stimuli are more frequent and more tuned close to the fovea. Model simulations after training have been performed in crossmodal conditions to assess the auditory and visual perception bias: visual stimuli were positioned at different azimuth (±10° from the fovea) coupled with an auditory stimulus at various audio-visual distances (±20°). The cue reliability has been altered by using visual stimuli with two different contrast levels. Model predictions are compared with behavioral data. Results show that model predictions agree with behavioral data, in a variety of conditions characterized by a different role of prior and likelihood. Finally, the effect of a different unimodal or crossmodal prior, re-learning, temporal correlation among input stimuli, and visual damage (hemianopia) are tested, to reveal the possible use of the model in the clarification of important multisensory problems.

Citation

Ursino, M., Cuppini, C., Magosso, E., Beierholm, U., & Shams, L. (2019). Explaining the Effect of Likelihood Manipulation and Prior through a Neural Network of the Audiovisual Perception of Space. Multisensory Research, 32(2), 111-144. https://doi.org/10.1163/22134808-20191324

Journal Article Type	Article
Acceptance Date	Jan 4, 2019
Online Publication Date	Feb 13, 2019
Publication Date	Jan 31, 2019
Deposit Date	Jan 8, 2019
Publicly Available Date	Feb 13, 2020
Journal	Multisensory Research
Print ISSN	2213-4794
Electronic ISSN	2213-4808
Publisher	Brill Academic Publishers
Peer Reviewed	Peer Reviewed
Volume	32
Issue	2
Pages	111-144
DOI	https://doi.org/10.1163/22134808-20191324