Multilingual Stance Detection in Tweets: The Catalonia Independence Corpus

Proceedings of the 12th Conference on Language Resources and Evaluation (LREC 2020), pages 1368–1375 Marseille, 11–16 May 2020 c⃝ European Language Resources Association (ELRA), licensed under CC-BY-NC 1368 Multilingual Stance Detection: The Catalonia Independence Corpus Elena Zotova1, Rodrigo Agerri1, Manuel Nu˜nez2, German Rigau1 1 IXA Group, HiTZ Centre, University of the Basque Country UPV/EHU, Donostia-San Sebastian, Spain 2 Intercom Strategys, Madrid, Spain zotova.el@gmail.com, rodrigo.agerri@ehu.eus, manuel.nunez@lunigtuk.com, german.rigau@ehu.eus Abstract Stance detection aims to determine the attitude of a given text with respect to a specific topic or claim. While stance detection has been fairly well researched in the last years, most the work has been focused on English. This is mainly due to the relative lack of annotated data in other languages. The TW-10 Referendum Dataset released at IberEval 2018 is a previous effort to provide multilingual stance-annotated data in Catalan and Spanish. Unfortunately, the TW-10 Catalan subset is extremely imbalanced. This paper addresses these issues by presenting a new multilingual dataset for stance detection in Twitter for the Catalan and Spanish languages, with the aim of facilitating research on stance detection in multilingual and cross-lingual settings. The dataset is annotated with stance towards one topic, namely, the independence of Catalonia. We also provide a semi-automatic method to annotate the dataset based on a categorization of Twitter users. We experiment on the new corpus with a number of supervised approaches, including linear classifiers and deep learning methods. Comparison of our new corpus with the with the TW-1O dataset shows both the benefits and potential of a well balanced corpus for multilingual and cross-lingual research on stance detection. Finally, we establish new state-of-the-art results on the TW-10 dataset, both for Catalan and Spanish. Keywords: Stance Detection, Text Categorization, Less-Resourced Languages 1. Introduction The rise of social media has given rise to the “fake news” phenomenon. According to the Fake News Challenge, “Fake news, defined by the New York Times as “a made-up story with an intention to deceive”1, often for a secondary gain, is arguably one of the most serious challenges facing the news industry today.”2 Determining the veracity of a given document or story, namely, whether it is fake or legitimate, is a very com- plex task, even for expert fact-checkers. Thus, previous work breaks down the fake news detection task in differ- ent stages, the first of which is establishing what other news sources are saying about the given document or story (whether they agree, disagree, etc. with the news story), namely, determining their stance with respect to that doc- ument or news story. Following this, the first stage of the Fake News Challenge was Stance Detection. This decision was supported by two main ideas: (i) a stance detection system should allow a human fact checker to enter a docu- ment (headline, message, claim, etc.) and retrieve the top documents from other news sources that agree, disagree or discuss the given document and, (ii) based on the previous step, it would be possible to build a “truth-labeling” sys- tem based on …

Content truncated to 500 words