Robust Retrieval Augmented Generation for Zero-shot Slot Filling

Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 1939–1949 November 7–11, 2021. c⃝2021 Association for Computational Linguistics 1939 Robust Retrieval Augmented Generation for Zero-shot Slot Filling Michael Glass, Gaetano Rossiello, Md Faisal Mahbub Chowdhury, Alfio Gliozzo IBM Research AI Thomas J. Watson Research Center, NY Abstract Automatically inducing high quality knowl- edge graphs from a given collection of docu- ments still remains a challenging problem in AI. One way to make headway for this prob- lem is through advancements in a related task known as slot filling. In this task, given an entity query in form of [ENTITY, SLOT, ?], a system is asked to ‘fill’ the slot by generat- ing or extracting the missing value exploiting evidence extracted from relevant passage(s) in the given document collection. The recent works in the field try to solve this task in an end-to-end fashion using retrieval-based lan- guage models. In this paper, we present a novel approach to zero-shot slot filling that ex- tends dense passage retrieval with hard neg- atives and robust training procedures for re- trieval augmented generation models. Our model reports large improvements on both T- REx and zsRE slot filling datasets, improving both passage retrieval and slot value genera- tion, and ranking at the top-1 position in the KILT leaderboard. Moreover, we demonstrate the robustness of our system showing its do- main adaptation capability on a new variant of the TACRED dataset for slot filling, through a combination of zero/few-shot learning. We re- lease the source code and pre-trained models1. 1 Introduction Slot filling is a sub-task of Knowledge Base Pop- ulation (KBP), where the goal is to recognize a pre-determined set of relations for a given entity and use them to populate infobox like structures. This can be done by exploring the occurrences of the input entity in the corpus and gathering infor- mation about its slot fillers from the context in which it is located. A slot filling system processes and indexes a corpus of documents. Then, when prompted with an entity and a number of relations, 1Our source code is available at: https://github. com/IBM/kgi-slot-filling Figure 1: Slot Filling task it fills out an infobox for the entity. Some slot fill- ing systems provide evidence text to explain the predictions. Figure 1 illustrates the slot filling task. Many KBP systems described in the literature commonly involve complex pipelines for named en- tity recognition, entity co-reference resolution and relation extraction (Ellis et al., 2015). In particu- lar, the task of extracting relations between entities from text has been shown to be the weakest com- ponent of the chain. The community proposed dif- ferent solutions to improve relation extraction per- formance, such as rule-based (Angeli et al., 2015), supervised (Zhang et al., 2017), or distantly su- pervised (Glass et al., 2018). However, all these approaches require a considerable human effort in creating hand-crafted rules, annotating training data, or building well-curated datasets for boot- strapping relation classifiers. Recently, pre-trained language models have been used …

Content truncated to 500 words