Annotation-Scheme Reconstruction for “Fake News” and Japanese Fake News Dataset

Proceedings of the 13th Conference on Language Resources and Evaluation (LREC 2022), pages 7226–7234 Marseille, 20-25 June 2022 © European Language Resources Association (ELRA), licensed under CC-BY-NC-4.0 7226 Annotation-Scheme Reconstruction for “Fake News” and Japanese Fake News Dataset Taichi Murayama*1, Shohei Hisada*2, Makoto Uehara2, Shoko Wakamiya2, Eiji Aramaki2 1SANKEN Osaka University, 2NARA Institute of Science and Technology, taichi88@sanken.osaka-u.ac.jp, {s-hisada, uehara.makoto.ug2, wakamiya, aramaki}@is.naist.jp Abstract Fake news provokes many societal problems; therefore, there has been extensive research on fake news detection tasks to counter it. Many fake news datasets were constructed as resources to facilitate this task. Contemporary research focuses almost exclusively on the factuality aspect of the news. However, this aspect alone is insufficient to explain “fake news,” which is a complex phenomenon that involves a wide range of issues. To fully understand the nature of each instance of fake news, it is important to observe it from various perspectives, such as the intention of the false news disseminator, the harmfulness of the news to our society, and the target of the news. We propose a novel annotation scheme with fine-grained labeling based on detailed investigations of existing fake news datasets to capture these various aspects of fake news. Using the annotation scheme, we construct and publish the first Japanese fake news dataset. The annotation scheme is expected to provide an in-depth understanding of fake news. We plan to build datasets for both Japanese and other languages using our scheme. Our Japanese dataset is published at https://hkefka385.github.io/dataset/fakenews-japanese/. Keywords: fake news, misinformation, disinformation, social media, computational social science, dataset 1. Introduction Fake news has caused significant damage to vari- ous fields of society, such as the economy, politics, and health problems. For example, during the 2016 U.S. presidential election, 529 different low-credibility statements (Jin et al., 2017) were spread on Twitter. Moreover, 25% of the news outlets that were linked from tweets, which were either fake or extremely bi- ased in supporting Trump or Clinton, potentially influ- enced the election (Bovet and Makse, 2019). Recently, the COVID-19 pandemic in 2020 resulted in the spread of disinformation and harmful content in the rapid in- flux of information, such as the relationship between the COVID-19 vaccine and infertility (Schraer, 2021). Fake news has become a significant crisis that threatens a wholesome society and the social media ecosystem. Previous studies have proposed various tasks to combat the social problems caused by the spread of fake news. For example, the fake news detection task aims to clas- sify whether the news content that is spread from news articles and social media posts is false. Additionally, many fake news datasets have been constructed as re- sources to facilitate the task, e.g., FakeNewsNet (Shu et al., 2020), Twitter16 (Ma et al., 2017), and CoAID (Cui and Lee, 2020). These existing studies on fake news detection and the corresponding dataset construction have focused almost exclusively on the factuality as- pect of the news – Can we fully understand “fake news” and various events it causes based on these datasets …

Content truncated to 500 words