Missing Counter-Evidence Renders NLP Fact-Checking Unrealistic for Misinformation

Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 5916 - 5936 December 7-11, 2022 ©2022 Association for Computational Linguistics Missing Counter-Evidence Renders NLP Fact-Checking Unrealistic for Misinformation Max Glockner1, Yufang Hou2, Iryna Gurevych1 1 Ubiquitous Knowledge Processing Lab (UKP Lab), Department of Computer Science and Hessian Center for AI (hessian.AI), Technical University of Darmstadt 2 IBM Research Europe, Ireland www.ukp.tu-darmstadt.de, yhou@ie.ibm.com Abstract Misinformation emerges in times of uncertainty when credible information is limited. This is challenging for NLP-based fact-checking as it relies on counter-evidence, which may not yet be available. Despite increasing interest in automatic fact-checking, it is still unclear if automated approaches can realistically refute harmful real-world misinformation. Here, we contrast and compare NLP fact-checking with how professional fact-checkers combat misin- formation in the absence of counter-evidence. In our analysis, we show that, by design, ex- isting NLP task definitions for fact-checking cannot refute misinformation as professional fact-checkers do for the majority of claims. We then define two requirements that the evi- dence in datasets must fulfill for realistic fact- checking: It must be (1) sufficient to refute the claim and (2) not leaked from existing fact-checking articles. We survey existing fact- checking datasets and find that all of them fail to satisfy both criteria. Finally, we perform ex- periments to demonstrate that models trained on a large-scale fact-checking dataset rely on leaked evidence, which makes them unsuitable in real-world scenarios. Taken together, we show that current NLP fact-checking cannot realistically combat real-world misinformation because it depends on unrealistic assumptions about counter-evidence in the data1. 1 Introduction According to van der Linden (2022), misinforma- tion is “false or misleading information masquerad- ing as legitimate news, regardless of intent”. Mis- information is dangerous as it can directly impact human behavior and have harmful real-world con- sequences such as the Pizzagate shooting (Fisher et al., 2016), interfering in the 2016 democratic US election (Bovet and Makse, 2019), or the promo- tion of false COVID-19 cures (Aghababaeian et al., 1Code provided at https://github.com/UKPLab/ emnlp2022-missing-counter-evidence Figure 1: A false claim from PolitiFact. It is unlikely to find counter-evidence. Fact-checkers refute the claim by disproving why it was made. 2020). Surging misinformation during the COVID- 19 pandemic, coined “infodemic” by WHO (Zaro- costas, 2020), exemplifies the danger coming from misinformation. To combat misinformation, jour- nalists from fact-checking organizations (e.g., Poli- tiFact or Snopes) conduct a laborious manual effort to verify claims based on possible harms and their prominence (Arnold, 2020). However, manual fact- checking cannot keep pace with the rate at which misinformation is posted and circulated. Auto- matic fact-checking has gained significant attention within the NLP community in recent years, with the goal of developing tools to assist fact-checkers in combating misinformation. For the past few years, NLP researchers have created a wide range of fact- checking datasets with claims from fact-checking organization websites (Vlachos and Riedel, 2014; Wang, 2017; Augenstein et al., 2019; Hanselowski et al., 2019; Ostrowski et al., 2021; Gupta and Srikumar, 2021; Khan et …

Content truncated to 500 words