Multisensory integration of redundant and complementary cues

Thesis

During multisensory integration, information from distinct sensory systems that refers to the same physical event is combined. For example, the sound and image that an individual generates as s/he interacts with the world, will provide the nervous system with multiple cues which can be integrated to estimate the individual’s position in the environment. However, the information that is perceived through different sensory pathways/systems can be qualitatively different. The information can be redundant and describe the same property of an event in a common reference frame (i.e., the image and sound referring to the individual’s location), or it can be complementary. Combining complementary information can be advantageous in that it extends the range and richness of the information available to the nervous system, but can also be superfluous and unnecessary to the task at hand – i.e. olfactory cues about the individuals perfume can increase the richness of the representation but not necessarily aid in localisation. Over the last century or so, a large body of research has focused on different aspects of multisensory interactions at both the behavioural and neural levels. It is currently unclear whether the mechanisms underlying multisensory interactions for both type of cue are similar or not. Moreover, the evidence for differences in behavioural outcome, dependent on the nature of the cue, is growing. Such cue property effects possibly reflect a processing heuristic for more efficient parsing of the vast amount of sensory information available to the nervous system at any one time.

The present thesis assesses the effects of cue properties (i.e., redundant or complementary) on multisensory processing and reports a series of experiments demonstrating that the nature of the cue, defined by the task of the observer, influences whether the cues compete for representation as a result of interacting, or whether instead multisensory information produces an optimal increase in reliability of the event estimate. Moreover, a bridging series of experiments demonstrate the key role of redundancy in inferring that two signals have a common physical cause and should be integrated, despite conflict in the cues. The experiments provide insights into the different strategies adopted by the nervous system and some tentative evidence for possible, distinct underlying mechanisms.

Authors: Hartcher-O'Brien, J

• Publication date: 2012
• Type of award: DPhil
• Level of award: Doctoral
• Awarding institution: Oxford University, UK
• Language: English
• Keywords: recalibration; cue combination; temporal estimates; multisensory integration; perception
• Subjects: Perception; Experimental psychology