On Unifying Misinformation Detection

Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 5479–5485 June 6–11, 2021. ©2021 Association for Computational Linguistics 5479 On Unifying Misinformation Detection Nayeon Lee§∗ Belinda Z. Li‡† Sinong Wang† Pascale Fung§ Hao Ma† Wen-tau Yih† Madian Khabsa† §Hong Kong University of Science and Technology †Facebook AI ‡MIT CSAIL nyleeaa@connect.ust.hk, mkhabsa@fb.com Abstract In this paper, we introduce UNIFIEDM2, a general-purpose misinformation model that jointly models multiple domains of misinfor- mation with a single, unified setup. The model is trained to handle four tasks: detect- ing news bias, clickbait, fake news and ver- ifying rumors. By grouping these tasks to- gether, UNIFIEDM2 learns a richer represen- tation of misinformation, which leads to state- of-the-art or comparable performance across all tasks. Furthermore, we demonstrate that UNIFIEDM2’s learned representation is help- ful for few-shot learning of unseen misinfor- mation tasks/datasets and model’s generaliz- ability to unseen events. 1 Introduction On any given day, 2.5 quintillion bytes of informa- tion are created on the Internet, a figure that is only expected to increase in the coming years (Marr, 2018). The internet has allowed information to spread rapidly, and studies have found that misin- formation spreads quicker and more broadly than true information (Vosoughi et al., 2018). It is thus paramount for misinformation detection ap- proaches to be able to adapt to new, emerging prob- lems in real time, without waiting for thousands of training examples to be collected. In other words, the generalizability of such systems is essential. Misinformation detection is not well-studied from a generalizability standpoint. Misinforma- tion can manifest in different forms and domains, i.e., fake news, clickbait, and false rumors, and previous literature has mostly focused on build- ing specialized models for a single domain (Rubin et al., 2016; Omidvar et al., 2018; Ma et al., 2018). (Even prior literature on multi-tasking for misin- formation (Kochkina et al., 2018) focuses more on ∗Work partially done while interning at Facebook AI. †Work partially done while working at Facebook AI. Figure 1: Architecture of our UNIFIEDM2 model using auxiliary tasks to boost performance on a sin- gle task, rather than on all tasks.) However, though these domains may differ in format (long articles vs. short headlines and tweets) and exact objective (“is this fake” vs. “is this clickbait”), they have the same ultimate goal of deceiving their readers. As a result, their content often exhibits similar linguistic characteristics, such as using a sensational style to incite curiosity or strong emotional responses from readers. Furthermore, models trained on multiple tasks are more robust and less prone to overfitting to spurious domain-specific correlations. Thus, uni- fying various domains of misinformation allows us to build a generalizable model that performs well across multiple domains/formats of misinforma- tion. In this work, we propose Unified Misinfo Model (UNIFIEDM2), a misinformation detection model that uses multi-task learning (Caruana, 1997; Mau- rer et al., 2016; Zhang and Yang, 2017) to train on different domains of misinformation. Through a comprehensive …

Content truncated to 500 words