All That’s ‘Human’ Is Not Gold: Evaluating Human Evaluation of Generated Text

Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing, pages 7282–7296 August 1–6, 2021. ©2021 Association for Computational Linguistics 7282 All That’s ‘Human’ Is Not Gold: Evaluating Human Evaluation of Generated Text Elizabeth Clark1 Tal August1 Sofia Serrano1 Nikita Haduong1 Suchin Gururangan1 Noah A. Smith1,2 1Paul G. Allen School of Computer Science & Engineering, University of Washington 2Allen Institute for Artificial Intelligence {eaclark7,taugust,sofias6,qu,sg01,nasmith}@cs.washington.edu Abstract Human evaluations are typically considered the gold standard in natural language gener- ation, but as models’ fluency improves, how well can evaluators detect and judge machine- generated text? We run a study assessing non- experts’ ability to distinguish between human- and machine-authored text (GPT2 and GPT3) in three domains (stories, news articles, and recipes). We find that, without training, evalua- tors distinguished between GPT3- and human- authored text at random chance level. We explore three approaches for quickly training evaluators to better identify GPT3-authored text (detailed instructions, annotated examples, and paired examples) and find that while eval- uators’ accuracy improved up to 55%, it did not significantly improve across the three do- mains. Given the inconsistent results across text domains and the often contradictory rea- sons evaluators gave for their judgments, we examine the role untrained human evaluations play in NLG evaluation and provide recom- mendations to NLG researchers for improv- ing human evaluations of text generated from state-of-the-art models. 1 Introduction Human-quality text has long been a holy grail for the output of natural language generation (NLG) systems, serving as an upper bound on their per- formance. Since we lack a good way of encoding many aspects of what constitutes human-quality output in an automated method, we often must rely on human evaluation for our models. Though eval- uations with end-users in an applied setting are en- couraged (Belz and Reiter, 2006), in practice, most human evaluations instead ask people to rate gener- ated text’s intrinsic quality (van der Lee et al., 2019; Howcroft et al., 2020). Sometimes the generated text is explicitly compared to human-authored text (e.g., Liu et al., 2016; Zellers et al., 2021; Zhang Figure 1: Excerpts from human evaluators’ explana- tions for why they believe a GPT3-generated story (also excerpted) was written by a human (left) or a machine (right). The evaluators point to a wide range of text attributes to make their decisions, sometimes using the same aspect of the text to come to opposite conclusions. et al., 2020), but even when no human-authored text is evaluated, evaluators implicitly compare the generated text to their knowledge of language and norms within specific domains. Evaluators are often asked to assess a text holis- tically, e.g., based on its overall quality, natural- ness, or humanlikeness (van der Lee et al., 2021; Howcroft et al., 2020), where the exact evaluation criteria is left to the discretion of the evaluator. Though other evaluations are broken down along specific dimensions of text quality (e.g., grammat- icality, coherence, etc.), Novikova et al. (2017, …

Content truncated to 500 words