TrClaim-19: The First Collection for Turkish Check-Worthy Claim Detection with Annotator Rationales

Proceedings of the 24th Conference on Computational Natural Language Learning, pages 386–395 Online, November 19-20, 2020. c⃝2020 Association for Computational Linguistics https://doi.org/10.18653/v1/P17 386 TrClaim-19: The First Collection for Turkish Check-Worthy Claim Detection with Annotator Rationales Yavuz Selim Kartal and Mucahid Kutlu Department of Computer Engineering TOBB University of Economics and Technology Ankara, Turkey {ykartal,m.kutlu}@etu.edu.tr Abstract Massive misinformation spread over Internet has many negative impacts on our lives. While spreading a claim is easy, investigating its ve- racity is hard and time consuming, Therefore, we urgently need systems to help human fact- checkers. However, available data resources to develop effective systems are limited and the vast majority of them is for English. In this work, we introduce TrClaim-19, which is the very first labeled dataset for Turkish check- worthy claims. TrClaim-19 consists of labeled 2287 Turkish tweets with annotator rationales, enabling us to better understand the character- istics of check-worthy claims. The rationales we collected suggest that claims’ topics and their possible negative impacts are the main factors affecting their check-worthiness. 1 Introduction In 2013, World Economic Forum (WEF) listed mas- sive digital misinformation as one of the top global risks likely to occur in 10 years1. Unfortunately, we witnessed many unpleasant incidents due to mis- information spread over Internet such as massive stock price changes2, gunfights3, and others. Since the start of COVID-19 pandemic, we have also ob- served many incidents showing the value of true information and how misinformation about health issues can be deadly (e.g., misusing disinfectants to prevent coronavirus after Donald Trump suggested injecting disinfectants as treatment4). 1http://reports.weforum.org/global- risks-2013 2www.reuters.com/article/net-us- usa-whitehouse-ap/hackers-send-fake- market-moving-ap-tweet-on-white-house- explosions-idUSBRE93M12Y20130423 3www.nytimes.com/2016/12/05/business/ media/comet-ping-pong-pizza-shooting- fake-news-consequences.html 4https://www.reuters.com/article/us- health-coronavirus-disinfectants- idUSKBN23C2P2 In order to prevent the negative outcomes of mis- information, many fact-checking websites emerged all over the world in the last decade (Cherubini and Graves, 2016). The fact-checking websites manu- ally investigate veracity of claims and share their findings with their readers. While they play an important role in the combat against misinforma- tion, their precious journalistic effort is not enough to reduce spread of misinformation and its nega- tive outcomes. While making a claim is so easy, investigating its veracity is highly time consum- ing, taking around one day (Hassan et al., 2017). Furthermore, Vosoughi et al. (2018) report that misinformation spread eight times faster than true information. Hence, we need effective solutions to help human fact-checkers and to reduce the nega- tive impact of misinformation. As Nakov et al. (2018) outline, the first task of a fact-checking system is to detect whether a state- ment contains a check-worthy claim or not. Con- sidering the massive amount of messages shared on social media platforms, check-worthy claim de- tection models help human fact-checkers to filter out unimportant claims and use their valuable time to detect veracity of the most important claims. A number of researchers worked on this problem (e.g., (Lespagnol et al., 2019; Hassan et al., 2017; Jara- dat et al., 2018)) and shared tasks for check-worthy claim detection have been organized (Nakov et al., 2018; Atanasova …

Content truncated to 500 words