Threat Scenarios and Best Practices to Detect Neural Fake News

Proceedings of the 29th International Conference on Computational Linguistics, pages 1233–1249 October 12–17, 2022. 1233 Threat Scenarios and Best Practices for Neural Fake News Detection Artidoro Pagnoni† and Martin Garciarena♢ and Yulia Tsvetkov† †Paul G. Allen School of CSE, University of Washington ♢ SRI International {artidoro, yuliats}@cs.washington.edu, martin.graciarena@sri.com Abstract In this work, we discuss different threat scenar- ios from neural fake news generated by state- of-the-art language models. Through our ex- periments, we assess the performance of gener- ated text detection systems under these threat scenarios. For each scenario, we also identify the minimax strategy for the detector that min- imizes its worst case performance. This con- stitutes a set of best practices that practitioners can rely on. In our analysis, we find that de- tectors are prone to shortcut learning (i.e., lack of out-of-distribution generalization) and dis- cuss approaches to mitigate this problem and improve detectors more broadly. Finally, we argue that strong detectors should be released along with new generators.1 1 Introduction During the COVID-19 pandemic, an overabun- dance of inaccurate information made it hard for people to find reliable guidance when they needed it, creating the first global infodemic. This widespread presence of online fake news poses a risk for the principles of the global information and communication space, which is considered to be a public good (Stiglitz et al., 1999). Widely available and reliable information is necessary for the func- tion of a democratic state and is seen by UNESCO as vital to ensure “public participation and civic space”.2 Neural generation systems unlock new possibili- ties for disinformation campaigns to generate large quantities of targeted content (Zellers et al., 2019; Radford et al., 2019; Keskar et al., 2019; Brown et al., 2020). Recent progress in pretrained lan- guage models enables the generation of fluent, orig- inal text that can be easily confused with human- written text. Progressively more complex use of 1Resources including generators, detectors, and generated and pristine news articles are made available at https:// github.com/artidoro/detect-gentext 2UNESCO report “Information as a public good” such language models can be achieved depending on the technical expertise and resources of the users (i.e., access to the language model, capability to run existing systems, to finetune them, or to develop a new generator and train it). This is a pressing problem, and it is reported that certain actors spend substantial monetary resources to spread misinfor- mation and influence public opinion, for example during the COVID 19 pandemic 3 and the US Pres- idential Elections of 2016 (Badawy et al., 2018; Stiff and Johansson, 2021). Although the major- ity of online disinformation is currently manually written (Vargo et al., 2018), the rapid progress of language generation systems along with their in- creased availability could attract these actors for whom the financial barriers do not pose problems. To prevent the spread of artificial and potentially malicious content online, it is necessary to develop automated detection systems that can distinguish human and generated text reliably. In this work, we discuss the problem …

Content truncated to 500 words