InfoSurgeon: Cross-Media Fine-grained Information Consistency Checking for Fake News Detection

Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing, pages 1683–1698 August 1–6, 2021. ©2021 Association for Computational Linguistics 1683 InfoSurgeon: Cross-Media Fine-grained Information Consistency Checking for Fake News Detection Yi R. Fung1, Chris Thomas2, Revanth Reddy1, Sandeep Polisetty3, Heng Ji1, Shih-Fu Chang2, Kathleen McKeown2, Mohit Bansal4, Avirup Sil5 1University of Illinois at Urbana-Champaign, 2Columbia University 3UMass Amherst, 4University of North Carolina at Chapel Hill, 5IBM 1{yifung2,revanth3,hengji}@illinois.edu 2{christopher.thomas,sc250,kathy}@columbia.edu 3spolisetty@umass.edu, 4mbansal@cs.unc.edu, 5avi@us.ibm.com Abstract To defend against neural system-generated fake news, an effective mechanism is urgently needed. We contribute a novel benchmark for fake news detection at the knowledge element level, as well as a solution for this task which incorporates cross-media consistency check- ing to detect the fine-grained knowledge el- ements making news articles misinformative. Due to training data scarcity, we also formu- late a novel data synthesis method by manip- ulating knowledge elements within the knowl- edge graph to generate noisy training data with specific, hard to detect, known inconsisten- cies. Our detection approach outperforms the state-of-the-art (up to 16.8% absolute accuracy gain), and more critically, yields fine-grained explanations.1 1 Introduction In recent years, generative neural network mod- els in natural language processing (Zellers et al., 2019) and computer vision (Choi et al., 2018) have become the frontier for malicious actors to con- trollably generate misinformation at scale. These realistic-looking AI-generated “fake news” have been shown to easily deceive humans, and it is, thus, critical for us to develop robust verification techniques against machine-generated fake news (Tan et al., 2020; Zellers et al., 2019; Kaliyar et al., 2020). Current misinformation detection approaches mainly focus on document-level fake news detection using lexical features and semantic embedding representations (Wang, 2017; Karimi et al., 2018; Tan et al., 2020). However, fake news is often generated based on manipulating (misus- ing, exaggerating, or falsifying) only a small part of the true information, namely the knowledge 1The code, data and resources related to the misinforma- tion detector are made publicly available at https://github. com/yrf1/InfoSurgeon for research purposes. elements (KEs, including entities, relations and events). Moreover, recent news oftentimes makes claims that do not have verified evidence yet, and evaluating the truthfulness of these real-time claims depends more on their consistency with other infor- mation conveyed in other data modalities. In this paper, we propose a new task: fine- grained, knowledge element-level cross-media in- formation consistency checking. The task involves treating the entire multimedia news article as one whole interconnected claim, where the goal is to de- tect misinformative KEs across the image, caption, and body text, as revealed by inconsistencies with respect to itself, or to background knowledge. This KE-level detection approach directly points out the fake pieces of information in the news, allowing for better explainability. Figure 1 shows an example where both the text and image provide complementary informa- tion about key argument roles of an event. We present the Information Surgeon (InfoSurgeon) model, which takes full advantage of state-of-the- …

Content truncated to 500 words