Audiovisual processing in Schizophrenia : neural responses in audiovisual speech interference and semantic priming

Abstract

Information processing in everyday life is to a large extent characterized by simultaneous input from different sensory modalities rather than from input of one single modality. To understand sensory processing, it is therefore vital to consider the multisensory aspect of everyday situations. Alterations in neural responses to unisensory stimuli have been reported extensively in schizophrenia in the past. The aim of the present work is the investigation of neurophysiological responses in multisensory processing and whether alterations exist in schizophrenia. To include two cognitive functions with a considerable influence on sensory processing, attention as well as semantic memory were incorporated in the experimental designs, which therefore comprise an audiovisual attention task in the EEG (study 1 and study 2) as well as a visual-to-auditory semantic priming paradigm in the MEG (study 3). The auditory speech interference paradigm was first applied in healthy subjects (study 1) and presented a video stream of three speakers who produced syllables, with one target syllable -occasionally produced by the center speaker- as the target syllable. As distracting input, flanking speakers produced no speech stimuli (No Interference Condition), audiovisual speech (Speech Interference Condition), auditory noise (Auditory Noise Interference Condition) or lip movements only (Lip Movement Interference Condition). The video was presented using an on-off flicker in different frequencies, 19 Hz for the two distracting flanking speakers, and 25 Hz for the to-be-attended center speaker, so Steady-State Visual Evoked Potentials (SSVEP) could be recorded in the EEG. Results indicated longer reaction times in response to distracting audiovisual speech, higher hit rates in the No Interference as compared to the Speech Interference and Auditory Noise Interference condition as well as enhanced distracting speaker induced SSVEPs for audiovisual speech, reflecting enhanced behavioural as well as neuronal correlates of distractibility towards natural audiovisual speech as compared to other distractors on the neuronal level. Subsequently, the paradigm was adapted and used in patients with schizophrenia (study 2). SSVEP power differences failed to reach significance, but behavioural data show that naturalistic speech is a stronger distractor than auditory noise, which parallels the behavioural findings in healthy subjects (study 1). The visual-to-auditory semantic priming study (study 3) involved an S1-S2 paradigm with naturalistic objects that were presented as pictures (S1) and subsequent sounds (S2) in a sample of patients with schizophrenia and matched control subjects (study 3). The semantic congruency of the object pairs was modulated and subjects were required to respond with an implicit judgment to the auditory S2. A behavioral priming effect, i.e. facilitated auditory object recognition, was found as indicated by significantly shorter reaction times on the auditory S2 for congruent as compared to the incongruent object pairs in all subjects. Furthermore, analysis of the MEG data showed reduced evoked power in patients with schizophrenia as compared to non-clinical control subjects between 30-40 Hz in the time window 20-80 msec as well as a significant statistic interaction for the factors group and condition. Additionally, correlations between 30-40 Hz power and different symptoms of the disease were found. Taken together, the findings of this thesis point to the relevance of early sensory processing in different mental actions and specifically, the correlation between MEG power and symptom strength in schizophrenia supports the notion of a possible impact for understanding dysfunctional neuronal functioning in schizophrenia.