Rich Knowledge Sources Bring Complex Knowledge Conflicts: Recalibrating Models to Reflect Conflicting Evidence

Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 2292 - 2307 December 7-11, 2022 ©2022 Association for Computational Linguistics Rich Knowledge Sources Bring Complex Knowledge Conflicts: Recalibrating Models to Reflect Conflicting Evidence Hung-Ting Chen Michael J.Q. Zhang Department of Computer ScienceEunsol Choi The University of Texas at Austin {hungtingchen, mjqzhang, eunsol}@utexas.edu Abstract Question answering models can use rich knowledge sources — up to one hundred re- trieved passages and parametric knowledge in the large-scale language model (LM). Prior work assumes information in such knowledge sources is consistent with each other, paying lit- tle attention to how models blend information stored in their LM parameters with that from retrieved evidence documents. In this paper, we simulate knowledge conflicts (i.e., where parametric knowledge suggests one answer and different passages suggest different an- swers) and examine model behaviors. We find retrieval performance heavily impacts which sources models rely on, and current models mostly rely on non-parametric knowledge in their best-performing settings. We discover a troubling trend that contradictions among knowledge sources affect model confidence only marginally. To address this issue, we present a new calibration study, where mod- els are discouraged from presenting any single answer when presented with multiple conflict- ing answer candidates in retrieved evidences. 1 Introduction Traditionally, QA models have relied on retrieved documents to provide provenance for their an- swers (Chen et al., 2017). Recent studies (Petroni et al., 2019) have shown that large language models are able to retain vast amounts of factual knowl- edge seen during pretraining, and closed-book QA systems (Roberts et al., 2020) build upon this foun- dation by memorizing facts from QA finetuning. Retrieval-based generation approaches (Izacard and Grave, 2021; Lewis et al., 2020) emerge as the best of both worlds – generating free-form answers from the question paired with retrieved evidence documents. They further combine these parametric knowledge sources with a large number of retrieved evidence documents, achieving state-of-the-art per- formances on open retrieval QA datasets (Joshi et al., 2017; Kwiatkowski et al., 2019). Parametric Knowledge (Facts memorized during training) Non-parametric Knowledge (Documents retrieved at inference time) Passage 1 …Norway set the record for most total medals at a single Winter Olympics with 39, surpassing the… Passage 2 …Norway was the most successful nation at the games with 39 total medals, setting a new record for the most medals won by a country at a single Winter Olympics. Passage 3 ...With 36 total medals, Germany set a record for most total medals at a Winter Olympics... 🤖 I have passages suggesting conflicting answers, thus I should abstain from answering! 👤 Which country won the most medals in winter olympics? Norway? Germany? the U.S? The U.S. team had a historic Winter Games, winning an unprecedented 37 medals. Figure 1: Models can use both parametric and non- parametric knowledge sources. In this example, the answer could be the U.S./Norway/Germany. We investi- gate for a given question which knowledge source was the most influential to output an answer. The model should …

Content truncated to 500 words