Misinfo Reaction Frames: Reasoning about Readers’ Reactions to News Headlines

Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics Volume 1: Long Papers, pages 3108 - 3127 May 22-27, 2022 c⃝2022 Association for Computational Linguistics Misinfo Reaction Frames: Reasoning about Readers’ Reactions to News Headlines Saadia Gabriel♠ Skyler Hallinan♠ Maarten Sap♢♡ Pemi Nguyen♠ Franziska Roesner♠ Eunsol Choi ♣ Yejin Choi♠♢ ♠Paul G. Allen School of Computer Science & Engineering, University of Washington ♣Department of Computer Science, The University of Texas at Austin ♢Allen Institute for Artificial Intelligence ♡Language Technologies Institute, Carnegie Mellon University {skgabrie,hallisky,peming,franzi,yejin}@cs.washington.edu, maartensap@cmu.edu, eunsol@utexas.edu Abstract Even to a simple and short news headline, read- ers react in a multitude of ways: cognitively (e.g. inferring the writer’s intent), emotionally (e.g. feeling distrust), and behaviorally (e.g. sharing the news with their friends). Such re- actions are instantaneous and yet complex, as they rely on factors that go beyond interpreting factual content of news. We propose Misinfo Reaction Frames (MRF), a pragmatic formalism for modeling how read- ers might react to a news headline. In contrast to categorical schema, our free-text dimensions provide a more nuanced way of understanding intent beyond being benign or malicious. We also introduce a Misinfo Reaction Frames cor- pus, a crowdsourced dataset of reactions to over 25k news headlines focusing on global crises: the Covid-19 pandemic, climate change, and cancer. Empirical results confirm that it is indeed possi- ble for neural models to predict the prominent patterns of readers’ reactions to previously un- seen news headlines. Additionally, our user study shows that displaying machine-generated MRF implications alongside news headlines to readers can increase their trust in real news while decreasing their trust in misinformation. Our work demonstrates the feasibility and im- portance of pragmatic inferences on news head- lines to help enhance AI-guided misinforma- tion detection and mitigation. 1 Introduction Many objects, persons, and experiences in the world are framed in terms of their potential role in supporting, harming, or enhancing people’s lives or interests. We can know that this is so if we know how to interpret expressions in which such things are evaluated... - Charles J. Fillmore (1976) Epidemics and cases of disease in the 21st century are "staged" Headline Reader "they are lying to us." Writer Intent Reader Perception "feel distrust" Reader Action "not trust the government" Likelihood of Spread "likely to share or read" (4/5) Perceived Label Gold Label "Misinfo" "Misinfo" Misinfo Reaction Frame Dimensions Figure 1: In a binary classification setup, the reaction of the reader is unclear. Here, we use Misinfo Reaction Frames to understand how the reader perceives and re- acts to the headline. Our pragmatic frames explain how a health or climate news article is interpreted as reliable or misinformation by readers by incorporating not only linguistic knowledge (e.g. emotions invoked by certain content words), but also knowledge of common social behaviors and domain-specific reasoning. We also in- clude fact-checked labels (gold label). Effectively predicting how a headline may in- fluence a reader requires knowledge of how read- ers perceive the intent behind real and …

Content truncated to 500 words