Fact-Checking Meets Fauxtography: Verifying Claims About Images

Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, pages 2099–2108, Hong Kong, China, November 3–7, 2019. c⃝2019 Association for Computational Linguistics 2099 Fact-Checking Meets Fauxtography: Verifying Claims About Images Dimitrina Zlatkova Sofia University “St. Kliment Ohridski” Sofia, Bulgaria dvzlatkova@uni-sofia.bg Preslav Nakov Qatar Computing Research Institute, HBKU Doha, Qatar pnakov@hbku.edu.qa Ivan Koychev Sofia University “St. Kliment Ohridski” Sofia, Bulgaria koychev@fmi.uni-sofia.bg Abstract The recent explosion of false claims in social media and on the Web in general has given rise to a lot of manual fact-checking initia- tives. Unfortunately, the number of claims that need to be fact-checked is several orders of magnitude larger than what humans can han- dle manually. Thus, there has been a lot of research aiming at automating the process. In- terestingly, previous work has largely ignored the growing number of claims about images. This is despite the fact that visual imagery is more influential than text and naturally ap- pears alongside fake news. Here we aim at bridging this gap. In particular, we create a new dataset for this problem, and we explore a variety of features modeling the claim, the image, and the relationship between the claim and the image. The evaluation results show sizable improvements over the baseline. We release our dataset, hoping to enable further re- search on fact-checking claims about images. 1 Introduction As social media become a bigger part of our daily lives, their influence over the way people think and make decisions increases. Inevitably, this has of- fered opportunities for fake content to arise and to spread faster than ever, e.g., recent research has shown that fake news spreads six time faster than real news (Vosoughi et al., 2018). Sometimes such content is created for pure entertainment or for fi- nancial gain from advertisement shown alongside the fake content, but more often and especially re- cently it has been used to spread disinformation, e.g., with the aim to influence political elections (Atanasov et al., 2019). To deal with the prob- lem, a number of manual fact-checking initiatives have been launched, but they remain insufficient to cope with the ever growing number of check- worthy claims. Thus, automated methods have been proposed as a more scalable solution. (a) A series of images show a Tesla vehicle in space. (b) Photograph shows a fire rainbow over Idaho. Figure 1: Examples of Real images and True claims. Recently, a growing number of claims have been about images. The word Fauxtography has been used to describe images, especially news photographs, that convey a questionable, or out- right false, sense of the events they seem to depict. The term was coined over a decade ago (Cooper, 2007), and there is growing research interest in the topic in the Computer Vision community (Bayar and Stamm, 2016; de Carvalho et al., 2016). Given the recent proliferation of fake news, and given that many of the questionable claims are about im- ages, it would be …

Content truncated to 500 words