How well does surprisal explain N400 amplitude under different experimental conditions?

Proceedings of the 24th Conference on Computational Natural Language Learning, pages 56–68 Online, November 19-20, 2020. c⃝2020 Association for Computational Linguistics https://doi.org/10.18653/v1/P17 56 Modeling Subjective Assessments of Guilt in Newspaper Crime Narratives Elisa Kreiss1,∗ Zijian Wang2,∗ 1Department of Linguistics 2Symbolic Systems Program Stanford University {ekreiss, zijwang, cgpotts}@stanford.edu Christopher Potts1,2 Abstract Crime reporting is a prevalent form of journal- ism with the power to shape public perceptions and social policies. How does the language of these reports act on readers? We seek to ad- dress this question with the SuspectGuilt Cor- pus of annotated crime stories from English- language newspapers in the U.S. For Suspect- Guilt, annotators read short crime articles and provided text-level ratings concerning the guilt of the main suspect as well as span-level an- notations indicating which parts of the story they felt most influenced their ratings. Sus- pectGuilt thus provides a rich picture of how linguistic choices affect subjective guilt judg- ments. We use SuspectGuilt to train and as- sess predictive models which validate the use- fulness of the corpus, and show that these mod- els benefit from genre pretraining and joint su- pervision from the text-level ratings and span- level annotations. Such models might be used as tools for understanding the societal effects of crime reporting. 1 Introduction News outlets around the world routinely report on crimes and alleged crimes, ranging from petty mis- demeanors to large-scale international criminal con- spiracies. Each of these reports will frame events in ways that shape reader perceptions, and these perceptions will in turn shape public perception of how much crime there is, who is responsible for crime, and what policy decisions should be made to address crime. It is therefore important to un- derstand how the language in these reports acts on readers, and there is clear value in developing NLP models that approximate these reader percep- tions at a large scale, as a tool for estimating the aggregate effects of crime reporting on society. To begin to address these needs, we present the SuspectGuilt Corpus of annotated crime stories ∗Equal contribution. Figure 1: The SuspectGuilt corpus highlighting inter- face. After participants responded to a question about the guilt of the main suspect in the report, they com- pleted this highlighting phase intended to provide in- sights into how they took themselves to be reasoning about the text. SuspectGuilt contains 1.8K stories with at least 5 participants responding to each. from English-language newspapers in the U.S.1 Each story in the corpus is multiply-annotated with participants’ assessments (on a continuous scale) of the guilt of the main suspect(s) and of the author’s belief in the guilt of the suspect(s). In addition, for each of these guilt-rating questions, the participants highlighted the spans of text in the story that they felt contributed to their decision (Figure 1). These additional annotations provide a window into the language that participants took themselves to be attending to as part of their personal verdicts, and thus they are especially useful for understanding how authors’ low-level …

Content truncated to 500 words