Neural mechanisms of face cue predictability and the integration of facial and acoustic cues in native-and nonnative-accented words

Abstract

This study examined the integration of face cue and native- and nonnative-accented English speech by manipulating the face cue predictability of a speaker’s accent (predictable: one accent (American-accent or Chinese-accented), not predictable: two accents (American-accented and Chinese-accented). Monolingual listeners were first familiarized with each speaker’s number and type of accent(s). Then, they completed an EEG-recorded auditory go/no-go animal decision task where the no-go items were critical words and nonwords. Listeners saw a face cue (speaker image) before speech onset and concurrently with the speech. Pre-speech ERP results revealed that listeners processed face cues differently based on face cue predictability. Post-speech ERP analyses revealed N400 lexicality effects for native-accented speech, and face cue predictability effects for nonnative-accented speech. No N400 effects were found for an audio-only experiment. This indicates that the monolingual listeners integrate face cues and auditory cues during real-time nonnative-accented speech processing, but not during native-accented speech processing.