The Authenticity Gap in Human Evaluation

Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 6056 - 6070 December 7-11, 2022 ©2022 Association for Computational Linguistics The Authenticity Gap in Human Evaluation Kawin Ethayarajh Stanford University kawin@stanford.edu Dan Jurafsky Stanford University jurafsky@stanford.edu Abstract Human ratings are the gold standard in NLG evaluation. The standard protocol is to collect ratings of generated text, average across anno- tators, and rank NLG systems by their average scores. However, little consideration has been given as to whether this approach faithfully cap- tures human preferences. Analyzing this stan- dard protocol through the lens of utility theory in economics, we identify the implicit assump- tions it makes about annotators. These assump- tions are often violated in practice, in which case annotator ratings cease to reflect their pref- erences. The most egregious violations come from using Likert scales, which provably re- verse the direction of the true preference in certain cases. We suggest improvements to the standard protocol to make it more theoretically sound, but even in its improved form, it can- not be used to evaluate open-ended tasks like story generation. For the latter, we propose a new human evaluation protocol called system- level probabilistic assessment (SPA). When hu- man evaluation of stories is done with SPA, we can recover the ordering of GPT-3 mod- els by size, with statistically significant results. However, when human evaluation is done with the standard protocol, less than half of the ex- pected preferences can be recovered (e.g., there is no significant difference between curie and davinci, despite using a highly powered test). 1 Introduction Human ratings are treated as the gold standard in NLG evaluation (Zhou et al., 2022). For exam- ple, say one wants to claim that that their NLG model X is better than the current state-of-the-art Y and Z for story generation. The standard proto- col is outcome-level absolute assessment (OAA): hire crowdworkers as annotators, collect individual ratings of a sample of stories generated by each model, and then claim that X is the best because its average rating is the highest (Celikyilmaz et al., 2020). There is inconsistency in how this is im- plemented in the literature: terms such as ‘text quality’ are often left undefined when instructing annotators (Howcroft et al., 2020) and different papers use different rating scales (Amidei et al., 2019). However, such criticism has been restricted to the implementations—little to no consideration has been given as to whether OAA can faithfully capture human preferences to begin with. We start by analyzing the standard evaluation protocol through the lens of utility theory from eco- nomics (§2). We find that OAA can only capture an annotator’s preferences under certain assumptions, which are unstated in the NLG literature and often violated in practice. In such cases, annotator ratings become an unfaithful reflection of their preferences (§3). For example, by framing ratings as utility estimates, we extend a result from Boutilier (2003) to prove that using the same scale is insufficient for aggregating ratings across annotators—they must …

Content truncated to 500 words