That is a Known Lie: Detecting Previously Fact-Checked Claims

Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 3607–3618 July 5 - 10, 2020. c⃝2020 Association for Computational Linguistics 3607 That is a Known Lie: Detecting Previously Fact-Checked Claims Shaden Shaar1, Nikolay Babulkov2, Giovanni Da San Martino1, Preslav Nakov1 1Qatar Computing Research Institute, HBKU, Doha, Qatar 2Sofia University, Sofia, Bulgaria {sshar, gmartino, pnakov}@hbku.edu.qa nbabulkov@gmail.com Abstract The recent proliferation of “fake news” has triggered a number of responses, most notably the emergence of several manual fact-checking initiatives. As a result and over time, a large number of fact-checked claims have been ac- cumulated, which increases the likelihood that a new claim in social media or a new statement by a politician might have already been fact- checked by some trusted fact-checking organi- zation, as viral claims often come back after a while in social media, and politicians like to repeat their favorite statements, true or false, over and over again. As manual fact-checking is very time-consuming (and fully automatic fact-checking has credibility issues), it is im- portant to try to save this effort and to avoid wasting time on claims that have already been fact-checked. Interestingly, despite the impor- tance of the task, it has been largely ignored by the research community so far. Here, we aim to bridge this gap. In particular, we formulate the task and we discuss how it relates to, but also differs from, previous work. We further create a specialized dataset, which we release to the research community. Finally, we present learning-to-rank experiments that demonstrate sizable improvements over state-of-the-art re- trieval and textual similarity approaches. 1 Introduction The year 2016 was marked by massive disinforma- tion campaigns related to Brexit and the US Presi- dential Elections. While false statements are not a new phenomenon, e.g., yellow press and tabloids have been around for decades, this time things were notably different in terms of scale and effectiveness thanks to social media platforms, which provided both a medium to reach millions of users and an easy way to micro-target specific narrow groups of voters based on precise geographical, demographic, psychological, and/or political profiling. Governments, international organizations, tech companies, media, journalists, and regular users launched a number of initiatives to limit the impact of the newly emerging large-scale weaponization of disinformation1 online. Notably, this included manual fact-checking initiatives, which aimed at debunking various false claims, with the hope to limit its impact, but also to educate the public that not all claims online are true. Over time, the number of such initiatives grew substantially, e.g., at the time of writing, the Duke Reporters’ Lab lists 237 active fact-checking orga- nizations plus another 92 inactive.2 While some organizations debunked just a couple of hundred claims, others such as Politifact,3 FactCheck.org,4 Snopes,5 and Full Fact6 have fact-checked thou- sands or even tens of thousands of claims. The value of these collections of resources has been recognized in the research community, and they have been used to train systems to perform automatic fact-checking (Popat et al., 2017; …

Content truncated to 500 words