Journal article
Combined predictive effects of sentential and visual constraints in early audiovisual speech processing
- Abstract:
- In language comprehension, a variety of contextual cues act in unison to render upcoming words more or less predictable. As a sentence unfolds, we use prior context (sentential constraints) to predict what the next words might be. Additionally, in a conversation, we can predict upcoming sounds through observing the mouth movements of a speaker (visual constraints). In electrophysiological studies, effects of visual constraints have typically been observed early in language processing, while effects of sentential constraints have typically been observed later. We hypothesized that the visual and the sentential constraints might feed into the same predictive process such that effects of sentential constraints might also be detectable early in language processing through modulations of the early effects of visual salience. We presented participants with audiovisual speech while recording their brain activity with magnetoencephalography. Participants saw videos of a person saying sentences where the last word was either sententially constrained or not, and began with a salient or non-salient mouth movement. We found that sentential constraints indeed exerted an early (N1) influence on language processing. Sentential modulations of the N1 visual predictability effect were visible in brain areas associated with semantic processing, and were differently expressed in the two hemispheres. In the left hemisphere, visual and sentential constraints jointly suppressed the auditory evoked field, while the right hemisphere was sensitive to visual constraints only in the absence of strong sentential constraints. These results suggest that sentential and visual constraints can jointly influence even very early stages of audiovisual speech comprehension.
- Publication status:
- Published
- Peer review status:
- Peer reviewed
Actions
Access Document
- Files:
-
-
(Preview, Version of record, pdf, 2.6MB, Terms of use)
-
- Publisher copy:
- 10.1038/s41598-019-44311-2
Authors
- Publisher:
- Nature Research
- Journal:
- Scientific Reports More from this journal
- Volume:
- 9
- Article number:
- 7870
- Publication date:
- 2019-05-27
- Acceptance date:
- 2019-05-14
- DOI:
- EISSN:
-
2045-2322
- Pmid:
-
31133646
- Language:
-
English
- Keywords:
- Pubs id:
-
pubs:1072694
- UUID:
-
uuid:3f3c5f5c-a0d9-447f-b609-e194ee93ad35
- Local pid:
-
pubs:1072694
- Source identifiers:
-
1072694
- Deposit date:
-
2019-12-09
Terms of use
- Copyright holder:
- Solberg Økland et al
- Copyright date:
- 2019
- Notes:
- Copyright © The Author(s) 2019. This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this license, visit http://creativecommons.org/licenses/by/4.0/.
- Licence:
- CC Attribution (CC BY)
If you are the owner of this record, you can report an update to it here: Report update to this record