A Survey on Automated Fact-Checking

A Survey on Automated Fact-Checking Zhijiang Guo∗, Michael Schlichtkrull∗, Andreas Vlachos Department of Computer Science and Technology University of Cambridge, UK {zg283,mss84,av308}@cam.ac.uk Abstract Fact-checking has become increasingly im- portant due to the speed with which both information and misinformation can spread in the modern media ecosystem. Therefore, researchers have been exploring how fact- checking can be automated, using techniques based on natural language processing, machine learning, knowledge representation, and data- bases to automatically predict the veracity of claims. In this paper, we survey automated fact-checking stemming from natural language processing, and discuss its connections to re- lated tasks and disciplines. In this process, we present an overview of existing datasets and models, aiming to unify the various definitions given and identify common concepts. Finally, we highlight challenges for future research. 1 Introduction Fact-checking is the task of assessing whether claims made in written or spoken language are true. This is an essential task in journalism, and is commonly conducted manually by dedicated organizations such as PolitiFact. In addition to external fact-checking, internal fact-checking is also performed by publishers of newspapers, mag- azines, and books prior to publishing in order to promote truthful reporting. Figure 1 shows an ex- ample from PolitiFact, together with the evidence (summarized) and the verdict. Fact-checking is a time-consuming task. To as- sess the claim in Figure 1, a journalist would need to search through potentially many sources to find job gains under Trump and Obama, evaluate the reliability of each source, and make a com- parison. This process can take professional fact- checkers several hours or days (Hassan et al., 2015; Adair et al., 2017). Compounding the prob- lem, fact-checkers often work under strict and ∗Equal contribution. tight deadlines, especially in the case of internal processes (Borel, 2016; Godler and Reich, 2017), and some studies have shown that less than half of all published articles have been subject to veri- fication (Lewis et al., 2008). Given the amount of new information that appears and the speed with which it spreads, manual validation is insufficient. Automating the fact-checking process has been discussed in the context of computational journal- ism (Flew et al., 2010; Cohen et al., 2011; Graves, 2018), and has received significant attention in the artificial intelligence community. Vlachos and Riedel (2014) proposed structuring it as a sequence of components—identifying claims to be checked, finding appropriate evidence, producing verdicts —that can be modeled as natural language pro- cessing (NLP) tasks. This motivated the develop- ment of automated pipelines consisting of subtasks that can be mapped to tasks well-explored in the NLP community. Advances were made possi- ble by the development of datasets, consisting of either claims collected from fact-checking web- sites, for example Liar (Wang, 2017), or purpose- made for research, for example, FEVER (Thorne et al., 2018a). A growing body of research is exploring the various tasks and subtasks necessary for the au- tomation of fact checking, and to meet the need for new methods to address emerging challenges. Early developments were surveyed in …

Content truncated to 500 words