Machine Learning Approach to Fact-Checking in West Slavic Languages

Proceedings of Recent Advances in Natural Language Processing, pages 973–979, Varna, Bulgaria, Sep 2–4, 2019. https://doi.org/10.26615/978-954-452-056-4_113 973 Machine Learning Approach to Fact-Checking in West Slavic Languages Pavel Pˇribáˇn1,2, Tomáš Hercig2, and Josef Steinberger1 1Department of Computer Science and Engineering, Faculty of Applied Sciences, University of West Bohemia, Czech Republic 2NTIS – New Technologies for the Information Society, Faculty of Applied Sciences, University of West Bohemia, Czech Republic {pribanp,tigi,jstein}@kiv.zcu.cz http://nlp.kiv.zcu.cz Abstract Fake news detection and closely-related fact-checking have recently attracted a lot of attention. Automatization of these tasks has been already studied for English. For other languages, only a few studies can be found (e.g. (Baly et al., 2018)), and to the best of our knowledge, no research has been conducted for West Slavic lan- guages. In this paper, we present datasets for Czech, Polish, and Slovak. We also ran initial experiments which set a baseline for further research into this area. 1 Introduction & Motivation Fake news is designed to incite agitation against an individual or a group of people. Its aim is to influence and manipulate public opinion on tar- geted topics. Fake news detection, including fact- checking, which can be used as the first step of a detection system, are currently receiving a lot of attention in the research community and journal- ism. This attention is apparent from the rise of fact-checking websites that verify mainly political claims (see the list of signatories of the code of principles of the International Fact-Checking Net- work1). Research related to these tasks is on the rise in a variety of fields, including natural lan- guage processing, machine learning, knowledge representation, databases, and journalism (Thorne and Vlachos, 2018). The automation of these tasks or their parts would greatly benefit journalism and perhaps help the public to verify the credibility of various me- dia. It is evident that fact-checking needs external 1https://ifcncodeofprinciples.poynter. org/signatories knowledge or detailed context. However, in or- der to achieve the goal of a robust automatic fact- checking system, we must first find a way how to evaluate such a system. For English, there are pub- licly available datasets that researchers can use to evaluate their systems. However, no systematic research has been conducted in West Slavic lan- guages yet; thus we establish a common ground for further research by providing large datasets for fact-checking in Czech, Polish, and Slovak lan- guages including initial experiments which reveal the complexity of the task. We set a baseline which uses standard machine learning approach, and set an upper bound which uses manually created ex- ternal knowledge. 2 Related Work This section presents a brief overview of related work, for a more detailed survey, please refer, for example to Thorne and Vlachos (2018). For the development of the first fact-checking systems, Vlachos and Riedel (2014) manually la- beled a dataset and defined fact-checking as the as- signment of a truth Boolean value to a claim made in a particular context. They also discussed base- line approaches to fact-checking. Wang (2017) presented a dataset …

Content truncated to 500 words