A Token-level Reference-free Hallucination Detection Benchmark for Free-form Text Generation

Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics Volume 1: Long Papers, pages 6723 - 6737 May 22-27, 2022 c⃝2022 Association for Computational Linguistics A Token-level Reference-free Hallucination Detection Benchmark for Free-form Text Generation Tianyu Liu1 2∗ Yizhe Zhang3 Chris Brockett4 Yi Mao4 Zhifang Sui1 Weizhu Chen4 Bill Dolan4 1 Peking University 2 Tencent Cloud Xiaowei 3 Meta AI 4 Microsoft Corporation {tianyu0421,szf}@pku.edu.cn , yizhe.zhang@hotmail.com {chrisbkt,maoyi,wzchen,billdol}@microsoft.com Abstract Large pretrained generative models like GPT- 3 often suffer from hallucinating non-existent or incorrect content, which undermines their potential merits in real applications. Existing work usually attempts to detect these halluci- nations based on a corresponding oracle refer- ence at a sentence or document level. How- ever ground-truth references may not be read- ily available for many free-form text genera- tion applications, and sentence- or document- level detection may fail to provide the fine- grained signals that would prevent fallacious content in real time. As a first step to address- ing these issues, we propose a novel token-level, reference-free hallucination detection task and an associated annotated dataset named HADES (HAllucination DEtection dataSet) 1. To cre- ate this dataset, we first perturb a large number of text segments extracted from English lan- guage Wikipedia, and then verify these with crowd-sourced annotations. To mitigate label imbalance during annotation, we utilize an iter- ative model-in-loop strategy. We conduct com- prehensive data analyses and create multiple baseline models. 1 Introduction Automatic text generation using neural natural lan- guage generation (NLG) systems is increasingly fluent and thus seemingly plausible in many real- world applications. Large-scale pretrained mod- els like GPT-3 (Brown et al., 2020) are proven to be powerful in understanding and performing free form text generation tasks at human-quality level with a few in-context examples, which dramati- cally reduces the manual labor needed in many text-based applications and services. Despite their ∗Work was done when Tianyu (intern) and Yizhe was at Microsoft. 1Code and data are provided in https://github. com/microsoft/HaDes great success, however, neural NLG systems using very large pre-trained models struggle to gener- ate factually accurate and trustworthy text (Devlin et al., 2019; Radford et al., 2019), and exhibit a propensity to hallucinate non-existent or incorrect content that is unacceptable in most user-oriented applications. This poses a major challenge for de- ploying production NLG systems with realtime generation, where post-examination is impossible. Existing work has sought to detect hallucination and quantitatively measure generation consistency against a provided reference. Such reference-based hallucination detection has been proposed for ab- stractive summarization (Maynez et al., 2020), ma- chine translation (Wang and Sennrich, 2020), data- to-text generation (Rebuffel et al., 2021), and im- age caption generation (Rohrbach et al., 2018). For many free-form text generation tasks, however, ref- erences are not readily available. For example, in a production NLG system such as a social chatbot using real-time response generation or a document auto-completion system, the generation model of- ten cannot pair its outputs with sufficient reference information, rendering reference-based methods less applicable: i) It may be …

Content truncated to 500 words