Where imagination meets sensation : mental imagery, perception and multisensory integration
Author: Berger, Christopher C
Date: 2016-06-03
Location: Hillarp Lecture Hall, Retzius väg 8, Karolinska Institutet, 171 77 Solna
Time: 09.00
Department: Inst för neurovetenskap / Dept of Neuroscience
View/ Open:
Thesis (2.250Mb)
Abstract
What happens if we imagine seeing something while we are listening to something? Will
it change what we hear? What happens if we imagine hearing something while we are
looking at something? Will it change what we see? In everyday perception, our brains
integrate the information provided to us by our different senses in order to form a coherent
perception of the world around us in a process referred to as multisensory integration.
However, sometimes the information provided to our senses arises from within, as is the case
when we imagine a sensory stimulus; for example, when you picture in your mind the face of
a loved one, or imagine how they sound when they say your name. The term mental imagery
is used to refer to these willed simulations of sensory stimuli in our minds. Empirical research
on mental imagery has demonstrated that there is a great deal of similarity in how we
consciously experience, and in how our brains process the sensory stimuli we imagine and
the sensory stimuli we perceive from the external world. However, whether our brains
integrate stimuli that are imagined in one sense and perceived in the other has never before
been explored. The main aim of this thesis was to investigate this possibility.
There were two main goals of the work comprising this thesis. First, to examine whether mental imagery is integrated with incoming sensory stimuli from a different sensory modality to change perception, and second, to examine the neural correlates of these mental imageryinduced changes in perception. Multisensory illusions have come to be a hallmark of multisensory integration as they are an easy and demonstrable way of measuring the integration of cross-modal sensory stimuli. Here, we have made use of classic multisensory illusions, and adapted them to investigate whether mental imagery in one sensory modality can integrate with veridical sensation in another sensory modality to produced fused multisensory percepts. We also used functional magnetic resonance imaging (fMRI) to examine whether brain areas related to multisensory integration of real stimuli were involved in multisensory integration of real and imagined stimuli.
By supplanting a real sensory stimulus with an imagined one in three different classic multisensory illusions, we found that imagined sensory stimuli were integrated with real sensory stimuli from a different sensory modality to change perception. Moreover, we found that these imagery-induced multisensory illusions followed the same spatial and temporal rules as classic multisensory illusions (Study I), as well as the unity-assumption rule of multisensory integration (Study II). Furthermore, we found that the neural correlates of a mental imagery-induced multisensory illusion were closely related to those known to be involved in integration of real multisensory stimuli (Study III). Lastly, consistent with what is known about adaptation to real multisensory stimuli, we found that repeated pairings of imagined and real stimuli from different sensory modalities lead to changes in future perception of the latter (Study IV). Together, these findings suggest that, indeed, what we imagine hearing can change what we see, and what we imagine seeing can change what we hear, affording mental imagery a larger role in multisensory perception than has ever before been considered.
There were two main goals of the work comprising this thesis. First, to examine whether mental imagery is integrated with incoming sensory stimuli from a different sensory modality to change perception, and second, to examine the neural correlates of these mental imageryinduced changes in perception. Multisensory illusions have come to be a hallmark of multisensory integration as they are an easy and demonstrable way of measuring the integration of cross-modal sensory stimuli. Here, we have made use of classic multisensory illusions, and adapted them to investigate whether mental imagery in one sensory modality can integrate with veridical sensation in another sensory modality to produced fused multisensory percepts. We also used functional magnetic resonance imaging (fMRI) to examine whether brain areas related to multisensory integration of real stimuli were involved in multisensory integration of real and imagined stimuli.
By supplanting a real sensory stimulus with an imagined one in three different classic multisensory illusions, we found that imagined sensory stimuli were integrated with real sensory stimuli from a different sensory modality to change perception. Moreover, we found that these imagery-induced multisensory illusions followed the same spatial and temporal rules as classic multisensory illusions (Study I), as well as the unity-assumption rule of multisensory integration (Study II). Furthermore, we found that the neural correlates of a mental imagery-induced multisensory illusion were closely related to those known to be involved in integration of real multisensory stimuli (Study III). Lastly, consistent with what is known about adaptation to real multisensory stimuli, we found that repeated pairings of imagined and real stimuli from different sensory modalities lead to changes in future perception of the latter (Study IV). Together, these findings suggest that, indeed, what we imagine hearing can change what we see, and what we imagine seeing can change what we hear, affording mental imagery a larger role in multisensory perception than has ever before been considered.
List of papers:
I. Berger, C. C. & Ehrsson, H. H. (2013). Mental Imagery Changes Multisensory Perception. Current Biology 23(14), 1367-72.
Fulltext (DOI)
Pubmed
View record in Web of Science®
II. Berger, C. C. & Ehrsson, H. H. What We Imagine Hearing Changes What We See. [Submitted]
III. Berger, C. C. & Ehrsson, H. H. (2014). The Fusion of Mental Imagery and Sensation in the Temporal Association Cortex. The Journal of Neuroscience 34(41), 13684-13692.
Fulltext (DOI)
Pubmed
View record in Web of Science®
IV. Berger, C. C. & Ehrsson, H. H. Rapidly Induced Auditory Plasticity by Imagined Visual Stimuli. [Manuscript]
I. Berger, C. C. & Ehrsson, H. H. (2013). Mental Imagery Changes Multisensory Perception. Current Biology 23(14), 1367-72.
Fulltext (DOI)
Pubmed
View record in Web of Science®
II. Berger, C. C. & Ehrsson, H. H. What We Imagine Hearing Changes What We See. [Submitted]
III. Berger, C. C. & Ehrsson, H. H. (2014). The Fusion of Mental Imagery and Sensation in the Temporal Association Cortex. The Journal of Neuroscience 34(41), 13684-13692.
Fulltext (DOI)
Pubmed
View record in Web of Science®
IV. Berger, C. C. & Ehrsson, H. H. Rapidly Induced Auditory Plasticity by Imagined Visual Stimuli. [Manuscript]
Institution: Karolinska Institutet
Supervisor: Ehrsson, H. Henrik
Issue date: 2016-05-12
Rights:
Publication year: 2016
ISBN: 978-91-7676-317-9
Statistics
Total Visits
Views | |
---|---|
Where ...(legacy) | 820 |
Where ... | 552 |
Total Visits Per Month
September 2023 | October 2023 | November 2023 | December 2023 | January 2024 | February 2024 | March 2024 | |
---|---|---|---|---|---|---|---|
Where ... | 3 | 3 | 1 | 1 | 8 | 8 | 1 |
File Visits
Views | |
---|---|
Thesis_Christopher_C_Berger.pdf(legacy) | 512 |
Thesis_Christopher_C_Berger.pdf | 338 |
null(legacy) | 2 |
Top country views
Views | |
---|---|
United States | 315 |
Sweden | 177 |
Germany | 102 |
United Kingdom | 91 |
Canada | 44 |
Philippines | 39 |
South Korea | 38 |
China | 35 |
Australia | 34 |
India | 28 |
Top cities views
Views | |
---|---|
Ashburn | 54 |
Degerfors | 30 |
Seoul | 30 |
Stockholm | 27 |
Wilmington | 23 |
Kiez | 22 |
Dublin | 20 |
London | 19 |
Woodbridge | 17 |
Hamburg | 14 |