Unraveling the Mystery of Artifacts in Machine Generated Text

Proceedings of the 13th Conference on Language Resources and Evaluation (LREC 2022), pages 6889–6898 Marseille, 20-25 June 2022 © European Language Resources Association (ELRA), licensed under CC-BY-NC-4.0 6889 Unraveling the Mystery of Artifacts in Machine Generated Text Jiashu Pu, Ziyi Huang, Yadong Xi, Guandan Chen, Weijie Chen, Rongsheng Zhang* Fuxi AI Lab, NetEase Inc., Hangzhou, China {pujiashu,huangziyi03,xiyadong,chenguandan,chenweijie05,zhangrongsheng}@corp.netease.com Abstract As neural Text Generation Models (TGM) have become more and more capable of generating text indistinguishable from human-written ones, the misuse of text generation technologies can have serious ramifications. Although a neural classifier often achieves high detection accuracy, the reason for it is not well studied. Most previous work revolves around studying the impact of model structure and the decoding strategy on ease of detection, but little work has been done to analyze the forms of artifacts left by the TGM. We propose to systematically study the forms and scopes of artifacts by corrupting text, replacing them with linguistic or statistical features, and applying the interpretable method of Integrated Gradients. Comprehensive experiments show artifacts a) primarily relate to token co-occurrence, b) feature more heavily at the head of vocabulary, c) appear more in content word than stopwords, d) are sometimes detrimental in the form of number of token occurrences, e) are less likely to exist in high-level semantics or syntaxes, f) manifest in low concreteness values for higher-order n-grams. Keywords: Natural Language Generation,Text categorisation,Corpus,Syntax,Semantics 1. Introduction In recent years, neural Text Generation Models (TGM) have grown by leaps and bounds with the support of big data and escalating computing power (Brown et al., 2020). State-of-the-art models can generate fluent and genuine-looking text (Gehrmann et al., 2019; Dou et al., 2021), thus we need to be especially wary of the abuse of text generation techniques, such as produc- ing fake news, impersonating others in email, phishing, etc. (Jawahar et al., 2020). Recently, Humans are worse at detecting synthetic text than Machines (Gehrmann et al., 2019; Solaiman et al., 2019) because Humans cannot properly perceive the difference in data distribution, but rely more on se- mantic errors or logical contradictions (Ippolito et al., 2020). This phenomenon has spawned much work in- vestigating the use of neural classifiers to detect syn- thetic text, most of which analyze the impact of vari- ables such as the structure of TGM, the decoding strat- egy, and the sentence length (Tay et al., 2020; Ippolito et al., 2020; Solaiman et al., 2019; Munir et al., 2021). Besides, a small amount of work studies TGM’s prop- erty through its left behind artifacts (detectable signa- tures that originate from the TGM). (Tay et al., 2020) reveals that TGMs of different decoding configurations leave distinguishable artifacts when generating text, proving artifacts are not correlated with word order. Another work (Ippolito et al., 2020) claims artifacts ex- ist in over-sampled high-likelihood words when TGM adopts decoding strategies of top-k sampling. Despite these sporadic findings, there is a lack of systematic analysis of the forms in which artifacts exist. To fill this void, we …

Content truncated to 500 words