Contrastive Error Attribution for Finetuned Language Models

Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics Volume 1: Long Papers, pages 11482–11498 July 9-14, 2023 ©2023 Association for Computational Linguistics Contrastive Error Attribution for Finetuned Language Models Faisal Ladhak1,2 Esin Durmus2 Tatsunori Hashimoto2 1Columbia University 2Stanford University faisal@cs.columbia.edu esindurmus@cs.stanford.edu thashim@stanford.edu Abstract Recent work has identified noisy and misanno- tated data as a core cause of hallucinations and unfaithful outputs in Natural Language Gener- ation (NLG) tasks. Consequently, identifying and removing these examples is a key open challenge in creating reliable NLG systems. In this work, we introduce a framework to iden- tify and remove low-quality training instances that lead to undesirable outputs, such as faith- fulness errors in text summarization. We show that existing approaches for error tracing, such as gradient-based influence measures, do not perform reliably for detecting faithfulness er- rors in NLG datasets. We overcome the draw- backs of existing error tracing methods through a new, contrast-based estimate that compares undesired generations to human-corrected out- puts. Our proposed method can achieve a mean average precision of 0.93 at detecting known data errors across synthetic tasks with known ground truth, substantially outperforming ex- isting approaches. Using this approach and re-training models on cleaned data leads to a 70% reduction in entity hallucinations on the NYT dataset and a 55% reduction in semantic errors on the E2E dataset. 1 Introduction Recent analyses of natural language generation sys- tems have identified that data errors are a key cause of failures ranging from unfaithfulness (Maynez et al., 2020) to bias (Torralba and Efros, 2011; Babaeianjelodar et al., 2020). While better data col- lection procedures (Yuan et al., 2021a; West et al., 2021) and noise-robust training methods (Kang and Hashimoto, 2020) can help address some of these problems, neither of these approaches serves as a complete solution. The large-scale datasets needed to train modern neural methods will inevitably con- tain at least a few annotation mistakes in these datasets, and some of these will affect even the most robust model training procedures. Data cleaning methods provide an alternative approach, where data errors are identified by trac- ing model errors back to the training dataset. This post-hoc approach allows practitioners to enforce desired properties such as faithfulness by repeat- edly identifying and removing rare data errors that cause undesired behavior. Existing work from the machine learning literature has proposed measur- ing the “influence” of training examples on gener- ated outputs as a way to trace such errors (Koh and Liang, 2017; Hara et al., 2019; Yuan et al., 2021b; Akyürek et al., 2022; Guu et al., 2023). However, these influence-based approaches are often brittle, and we find that they fail in com- plex, real-world tasks such as text summarization or data-to-text generation. In a synthetic evalua- tion inspired by prior work in the memorization literature (Carlini et al., 2019), we inject targeted hallucinations in the training data and evaluate er- ror tracing methods on how well they identify these errors and reduce downstream hallucination. We show that existing …

Content truncated to 500 words