Categorising Fine-to-Coarse Grained Misinformation: An Empirical Study of the COVID-19 Infodemic

Proceedings of Recent Advances in Natural Language Processing, pages 556–567 Varna, Sep 4–6, 2023 https://doi.org/10.26615/978-954-452-092-2_061 556 Categorising Fine-to-Coarse Grained Misinformation: An Empirical Study of the COVID-19 Infodemic Ye Jiang, Xingyi Song, Carolina Scarton, Iknoor Singh, Ahmet Aker, Kalina Bontcheva University of Sheffield, Sheffield, United Kingdom {ye.jiang, x.song, c.scarton, i.singh, ahmet.aker, k.bontcheva}@sheffield.ac.uk Abstract The spread of COVID-19 misinformation on social media became a major challenge for citizens, with negative real-life consequences. Prior research focused on detection and/or anal- ysis of COVID-19 misinformation. However, fine-grained classification of misinformation claims has been largely overlooked. The novel contribution of this paper is in introducing a new dataset1 which makes fine-grained distinc- tions between statements that assert, comment or question on false COVID-19 claims. This new dataset not only enables social behaviour analysis but also enables us to address both evidence-based and non-evidence-based misin- formation classification tasks. Lastly, through leave claim out cross-validation, we demon- strate that classifier performance on unseen COVID-19 misinformation claims is signifi- cantly different, as compared to performance on topics present in the training data. 1 Introduction For the majority of citizens, social media be- came the primary source of information during the COVID-19 pandemic (Sharma et al., 2020; Zhou et al., 2021). While social media allowed citizens to seek information in a more timely manner, it also resulted in an ‘infodemic’ (WHO, 2020) of misin- formation which has caused significant harms. Therefore, while independent fact-checkers (e.g., International Fact-Checking Network IFCN2) played a vital role, they increasingly need AI mod- els (Zeng et al., 2021) to help scale up and optimise the fact-checking workflows. Such models, how- ever, have been trained primarily on datasets of political and other non-COVID-19 misinformation, 1The dataset and the annotation codebook are available at https://doi.org/10.5281/zenodo.8131933. 2https://www.poynter.org/ifcn/ (Accessed on Feb 1, 2023) which has impacted their accuracy in detecting and classifying COVID-19 false claims. Prior studies of COVID-19 misinformation fo- cused mainly on misinformation detection (Hayawi et al., 2022; Gupta et al., 2021; Hossain et al., 2020), the social engagement with fake news on websites and social platforms (Cui and Lee, 2020), and the ways that misinformation is countered in tweets (Micallef et al., 2020). However, they have largely overlooked the wider online debates about COVID-19 misinformation, such as the conversa- tional threads around false COVID-19 claims and the questions and comments made as part of these. It is absolutely crucial for fact-checkers to have at their disposal models that not only flag misinforma- tion, but can also flag the comments and questions raised in online debates around false claims, so they can address them in debunks. In particular, this paper aims to address three re- search questions: RQ1: Which social media posts are propagating, questioning or commenting about a false claim? RQ2: Does the volume of tweets de- bunking a misinformation claim correlate with the volume of misinformation tweets? RQ3: What are the different kinds of COVID-19 misinformation spreading online? The novel contributions are: 1. A large dataset of COVID-19 tweets that are discussing IFCN fact-checked …

Content truncated to 500 words