Automatic Detection of Generated Text is Easiest when Humans are Fooled

Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 1808–1822 July 5 - 10, 2020. c⃝2020 Association for Computational Linguistics 1808 Automatic Detection of Generated Text is Easiest when Humans are Fooled Daphne Ippolito†‡∗ daphnei@seas.upenn.edu Daniel Duckworth‡* duckworthd@google.com Chris Callison-Burch†‡ ccb@seas.upenn.edu Douglas Eck‡ deck@google.com Abstract Recent advancements in neural language mod- elling make it possible to rapidly generate vast amounts of human-sounding text. The ca- pabilities of humans and automatic discrimi- nators to detect machine-generated text have been a large source of research interest, but hu- mans and machines rely on different cues to make their decisions. Here, we perform care- ful benchmarking and analysis of three popu- lar sampling-based decoding strategies—top- k, nucleus sampling, and untruncated random sampling—and show that improvements in de- coding methods have primarily optimized for fooling humans. This comes at the expense of introducing statistical abnormalities that make detection easy for automatic systems. We also show that though both human and automatic detector performance improve with longer ex- cerpt length, even multi-sentence excerpts can fool expert human raters over 30% of the time. Our findings reveal the importance of using both human and automatic detectors to assess the humanness of text generation systems. 1 Introduction State-of-the-art generative language models are now capable of producing multi-paragraph ex- cerpts that at a surface level are virtually indis- tinguishable from human-written content (Zellers et al., 2019; Radford et al., 2019; Adelani et al., 2020). Often, only subtle logical fallacies or id- iosyncrasies of language give away the text as machine-generated, errors that require a close reading and/or domain knowledge for humans to detect. Deceptive text, whether human- or machine- generated, has entered the sphere of public con- cern (Cooke, 2018). It propogates quickly (Vosoughi et al., 2018), sets political agendas ∗Equal contribution, ‡Google, †University of Pennsylva- nia (Vargo et al., 2018), influences elections (Allcott and Gentzkow, 2017), and undermines user trust (Wang et al., 2012; Song et al., 2015). Recently, Adelani et al. (2020) have shown that automati- cally generated reviews are perceived to be as flu- ent as human-written ones. As generative tech- nology matures, authors, well-meaning or other- wise, will increasingly employ it to augment and accelerate their own writing. It is more impera- tive now than ever for both humans and automated systems to be able to detect and identify machine- generated texts in the wild. However, there has thus been little inquiry into the textual proper- ties that cause humans to give generated text high human-like ratings compared to those that cause automatic systems to rate it highly. To speak of texts produced by language mod- els, we must first consider how these texts are generated. A neural language model encodes a probability distribution over the next word in a sequence given the previous words.1 A decod- ing strategy is an algorithm that generates se- quences from a language model by determining how words should get selected from this distribu- tion. The field has largely moved toward prob- abilistic decoding …

Content truncated to 500 words