Using surprisal and fMRI to map the neural bases of broad and local contextual prediction during natural language comprehension

Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021, pages 3786–3798 August 1–6, 2021. ©2021 Association for Computational Linguistics 3786 Using surprisal and fMRI to map the neural bases of broad and local contextual prediction during natural language comprehension Shohini Bhattasali and Philip Resnik Linguistics/UMIACS University of Maryland College Park, MD {shohini, resnik}@umd.edu Abstract Context guides comprehenders’ expectations during language processing, and information- theoretic surprisal is commonly used as an in- dex of cognitive processing effort. However, prior work using surprisal has considered only within-sentence context, using n-grams, neural language models, or syntactic structure as con- ditioning context. In this paper, we extend the surprisal approach to use broader topical con- text, investigating the influence of local and topical context on processing via an analysis of fMRI time courses collected during naturalis- tic listening. Lexical surprisal calculated from ngram and LSTM language models is used to capture effects of local context; to capture the effects of broader context a new metric based on topic models, topical surprisal, is intro- duced. We identify distinct patterns of neural activation for lexical surprisal and topical sur- prisal. These differing neuro-anatomical cor- relates suggest that local and broad contextual cues during sentence processing recruit differ- ent brain regions and that those regions of the language network functionally contribute to processing different dimensions of contex- tual information during comprehension. More generally, our approach adds to a growing lit- erature using methods from computational lin- guistics to operationalize and test hypotheses about neuro-cognitive mechanisms in sentence processing. 1 Introduction Narratives unfold over time and comprehenders in- crementally process words and sentences. In order to understand the current word and sentence, we have to integrate current input with the information from the previous context. In characterizing this process, the notion of sur- prisal from information theory has been preva- lent in psycholinguistic modeling, following the work of Hale (2001) and Levy (2008). Surprisal operationalizes how unexpected a word is as its pointwise information content given prior context, −log P(wi|w1...wi−1). Generally the theory of sur- prisal, as applied in the study of human language comprehension, proposes that probabilistic predic- tions made by comprehenders yield variability in word-by-word processing difficulty: when surprisal is high, the current word is unexpected and cogni- tive processing effort increases accordingly. This linking has been validated in prior work connect- ing surprisal with measurable reflexes of cognitive effort, using probabilities conditioned on lexical (sequential) and syntactic contexts (e.g. Brennan et al., 2016; Lopopolo et al., 2017; Brennan and Hale, 2019; Shain et al., 2020). In this paper, we extend the surprisal paradigm beyond prior work looking only at local context, to investigate the influence of broader contextual in- formation during incremental sentence processing. As an illustration, consider examples (a) and (b), which illustrate that, while a word might be ex- tremely difficult to predict given the immediate local context, it might be less unexpected/more predictable given the broader topic under discus- sion. The actual word to be predicted here is China, which is not …

Content truncated to 500 words