Is GPT-3 Text Indistinguishable from Human Text? Scarecrow: A Framework for Scrutinizing Machine Text

Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics Volume 1: Long Papers, pages 7250 - 7274 May 22-27, 2022 c⃝2022 Association for Computational Linguistics Is GPT-3 Text Indistinguishable from Human Text? SCARECROW: A Framework for Scrutinizing Machine Text Yao Dou∗† Maxwell Forbes∗† Rik Koncel-Kedziorski† Noah A. Smith†‡ Yejin Choi†‡ †Paul G. Allen School of Computer Science & Engineering, University of Washington ‡Allen Institute for AI {douy,mbforbes,nasmith,yejin}@cs.washington.edu kedzior@uw.edu Abstract Modern neural language models can produce remarkably fluent and grammatical text. So much, in fact, that recent work by Clark et al. (2021) has reported that conventional crowd- sourcing can no longer reliably distinguish be- tween machine-authored (GPT-3) and human- authored writing. As errors in machine gener- ations become ever subtler and harder to spot, it poses a new challenge to the research com- munity for robust machine text evaluation. We propose a new framework called SCARE- CROW for scrutinizing machine text via crowd annotation. To support the broad range of real machine errors that can be identified by laypeo- ple, the ten error categories of SCARECROW— such as redundancy , commonsense errors , and incoherence —are identified through sev- eral rounds of crowd annotation experiments without a predefined ontology. We then use SCARECROW to collect over 41k error spans in human-written and machine- generated paragraphs of English language news text. We isolate factors for detailed analysis, including parameter count, training data, and various decoding-time configura- tions. Our approach successfully quantifies measurable gaps between human authored text and generations from models of several sizes, including fourteen configurations of GPT-3. In addition, our analysis unveils new insights, with detailed rationales provided by laypeople, e.g., that the commonsense capabilities have been improving with larger models while math capabilities have not, and that the choices of simple decoding hyperparameters can make re- markable differences on the perceived quality of machine text. We release our training mate- rial, annotation toolkit and dataset at https: //yao-dou.github.io/scarecrow/. 1 Introduction Clark et al. (2021) demonstrated the challenges of human evaluation in the era of GPT-3 (Brown ∗Equal contribution Off-Prompt The long-rumored Apple car might finally become a reality. Prompt (human-authored) According to the Financial Times, Apple's been talking to "a small group of contract manufacturers to explore making an electric vehicle," which would ostensibly be an autonomous car. All this does sound like the loose ends of Apple's CarPlay rollout: hiring 1,200 engineers for the iOS team, building the CarPlay-specific testing track, developing a Lincoln Navigator, then poaching Burberry’s head of product design to lead the integration of software and hardware. WWDC 2015 We know what you're thinking: Another Monday? Continuation written by GPT-3 DaVinci The most likely meaning of “track” in this context is a driving area, which doesn’t make sense for CarPlay. Apple would develop their own car, not make a Lincoln Navigator, which already exists. Burberry’s head of product design wouldn't have the technical expertise needed for this particular job. While Apple CarPlay is also about cars, this isn’t actually relevant. This …

Content truncated to 500 words