Neural Deepfake Detection with Factual Structure of Text

Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing, pages 2461–2470, November 16–20, 2020. c⃝2020 Association for Computational Linguistics 2461 Neural Deepfake Detection with Factual Structure of Text Wanjun Zhong1∗, Duyu Tang2, Zenan Xu1∗, Ruize Wang3, Nan Duan2, Ming Zhou2 Jiahai Wang1 and Jian Yin1 1 The School of Data and Computer Science, Sun Yat-sen University. Guangdong Key Laboratory of Big Data Analysis and Processing, Guangzhou, P.R.China 2 Microsoft Research 3 Fudan University, Shanghai, P.R.China {zhongwj25@mail2,xuzn@mail2}.sysu.edu.cn {wangjiah@mail,issjyin@mail}.sysu.edu.cn {dutang,nanduan,mingzhou}@microsoft.com rzwang18@fudan.edu.cn Abstract Deepfake detection, the task of automatically discriminating machine-generated text, is in- creasingly critical with recent advances in nat- ural language generative models. Existing ap- proaches to deepfake detection typically rep- resent documents with coarse-grained repre- sentations. However, they struggle to cap- ture factual structures of documents, which is a discriminative factor between machine- generated and human-written text according to our statistical analysis. To address this, we propose a graph-based model that utilizes the factual structure of a document for deep- fake detection of text. Our approach repre- sents the factual structure of a given document as an entity graph, which is further utilized to learn sentence representations with a graph neural network. Sentence representations are then composed to a document representation for making predictions, where consistent re- lations between neighboring sentences are se- quentially modeled. Results of experiments on two public deepfake datasets show that our approach significantly improves strong base models built with RoBERTa. Model analy- sis further indicates that our model can dis- tinguish the difference in the factual structure between machine-generated text and human- written text. 1 Introduction Nowadays, unprecedented amounts of online mis- information (e.g., fake news and rumors) spread through the internet, which may misinform peo- ple’s opinions of essential social events (Faris et al., 2017; Thorne et al., 2018; Goodrich et al., 2019; Kry´sci´nski et al., 2019). Recent advances in neural generative models, such as GPT-2 (Radford et al., ∗ Work done while this author was an intern at Microsoft Research. … The law currently imposes a 100 million euro ($113 million) fine on Iran in case of a violation, but under the new bill, if Saudis violate the arms embargo, Tehran will be handed a 2 billion euros fine. … Zarif: No nuclear enrichment prohibition by Iran Neural Fake News April 15, 2019 Figure 1: An example of machine-generated fake news. We can observe that the factual structure of entities ex- tracted by named entity recognition is inconsistent. 2019), make the situation even severer as their abil- ity to generate fluent and coherent text may enable adversaries to produce fake news. In this work, we study deepfake detection of text, to automat- ically discriminate machine-generated text from human-written text. Previous works on deepfake detection of text are dominated by neural document classification mod- els (Bakhtin et al., 2019; Zellers et al., 2019; Wang et al., 2019; Vijayaraghavan et al., 2020). They typically tackle the problem with coarse-grained document-level evidence such as dense vectors learned by neural encoder and traditional features …

Content truncated to 500 words