IRMA: the 335-million-word Italian coRpus for studying MisinformAtion

Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics, pages 2339–2349 May 2-6, 2023 ©2023 Association for Computational Linguistics IRMA: the 335-million-word Italian coRpus for studying MisinformAtion Fabio Carrella School of Psychological Science University of Bristol fabio.carrella@bristol.ac.uk Alessandro Miani Institute of Work and Organizational Psychology University of Neuchâtel alessandro.miani@unine.ch Stephan Lewandowsky School of Psychological Science University of Bristol stephan.lewandowsky@bristol.ac.uk Abstract The dissemination of false information on the internet has received considerable attention over the last decade. Misinformation often spreads faster than mainstream news, thus mak- ing manual fact checking inefficient or, at best, labor-intensive. Therefore, there is an increas- ing need to develop methods for automatic detection of misinformation. Although re- sources for creating such methods are available in English, other languages are often under- represented in this effort. With this contribu- tion, we present IRMA, a corpus containing over 600,000 Italian news articles (335+ mil- lion tokens) collected from 56 websites clas- sified as ‘untrustworthy’ by professional fact- checkers. The corpus is freely available and comprises a rich set of text- and website-level data, representing a turnkey resource to test hypotheses and develop automatic detection al- gorithms. It contains texts, titles, and dates (from 2004 to 2022), along with three types of semantic measures (i.e., keywords, topics at three different resolutions, and LIWC lex- ical features). IRMA also includes domain- specific information such as source type (e.g., political, health, conspiracy, etc.), quality, and higher-level metadata, including several met- rics of website incoming traffic that allow to investigate user online behavior. IRMA con- stitutes the largest corpus of misinformation available today in Italian, making it a valid tool for advancing quantitative research on untrust- worthy news detection and ultimately helping limit the spread of misinformation.1 1 Introduction Over the last decade, there has been an increase in worry about misinformation, which has led to nu- merous studies (e.g., Lazer et al. 2018; Pennycook 1IRMA is freely available at https://osf.io/rywp4/. and Rand 2019; Roozenbeek et al. 2020). This level of focus is justified by the threat that misinforma- tion poses to individuals, institutions, and society in an increasingly digitalized world. Helped by social media capillarity and a lack of gatekeeping, mis- information is eroding long-standing institutional barriers, compromising democratic processes, as happened during the last US presidential elections (Allcott and Gentzkow, 2017; Dave et al., 2021), and producing serious sociopolitical uncertainty, as in the examples of global warming and COVID-19 vaccines (van der Linden et al. 2017; Loomba et al. 2021). Currently, there are two main approaches used to detect misinformation online: manual and auto- matic. The first relies on human effort, mostly rep- resented by fact-checking services that employ ex- perts to manually verify the accuracy of claims, ar- ticles, and entire websites. The second is based on the identification of particular textual content fea- tures, usually performed through natural language processing (NLP) tools, e.g., deep learning models. Because misinformation spreads alarmingly faster than reliable news (Vosoughi et al. 2018; Gravino et al. 2022), …

Content truncated to 500 words